
DOCUMENT RESUME 



BO 251 406 



SP 023 Oil 



AUTHOR 
TITLE 

INSTITUTION 

SPONS AGENCY 
REPORT NO 
PUB DATE 
CONTRACT 
NOTE 

AVAILABLE FROM 



PUB TYPE 



EDRS PRICE 
DESCRIPTORS 



Nelli, Elizabeth; Nutter, Norma 

A Model £or Evaluating Teacher Education Programs. 

Teacher Education Monograph No. 2. 

ERIC Clearinghouse on Teacher Education, Washington, 

D.C. 

National Inst. o£ Education (ED), Washington, DC. 

ISBN-0-89333-032-9 

Feb 84 

400-83-0022 

83p. 

ERIC Clearinghouse on Teacher Education, 1 Dupont 
Circle, NV:, Suite 610, Washington, DC 20036 
($6.50). 

Information Analyses - ERIC Information Analysis 
Products (071) — Guides * Non-Classroom Use (055) — 
Reports - Descriptive (141) 

HF01/PC04 Plus Postage. 

^Evaluation Criteria; ^Evaluation Methods; Higher 
Education; ^Institutional Characteristics; Preservice 
Teacher Education; Program Effectiveness; ^Program 
Evaluation; *Sel£ Evaluation (Groups); Student 
Characteristics; Teacher Characteristics; Teacher 
Education Curriculum; *Teacher Education Programs; 
Teacher Educators 



ABSTRACT 

Prior to presenting a model for evaluating teacher 
education progams, it is pointed out that five major issues must be 
settled beilore attempting an evaluation: (1) the evaluation's 
purpose; (2) clients or audience; (3) specific questions to 
investigate; (4) use of an internal or external evaluator or a 
combination; and (5) dissemination of the evaluation and types of 
decisions to be made based on its results. The matrix of the model 
includes the following elements: (1) the philosophy and basic 
purposes of the program; (2) objectives or intended outcomes of the 
evaluation; (3) collection and assessment of data; (4) 
characteristics of students, faculty, college policy, program budget, 
and administration; (5) interactions between mentors and students in 
the actual working curriculum; (6) outcomes attributable to the 
program; (7) external factors impinging on the program; (8) standards 
against which the program's quality or value are measured; and (9) 
judgments on the quality, value, or effectiveness of the program and 
its components. These elements, their interpretations, and their 
interrelations are described and discussed. Four examples are given 
demonstrating how the model might be applied in actual evaluations. 
(JD) 
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FOREWORD 



Recent concerns about the effectiveness of our schools voiced by 
many citizens and policy makers offer challenges as well to teacher 
education. The impact of parents and peer groups on the learning of 
our youth is well recognized. And the conditions of professional prac- 
tice in schools exert a powerful influence on what teachers can do. 
But neither of these realities should obscure the impprtant potential 
of the classroom teacher for creating quality learning opportunities 
in our schools. 

Offering preparation progi-ams that help tomorrow's teachers 
make a positive diffei-ence in school learnings is a demanding and often 
frustrating task. Certainly, it is a complex undertaking, and its com- 
plexity increases as Americans* demands upon public schools bur- 
geon. Teacher educators face the difficulties of defining effective 
teaching in the context of today's varied classrooms; of inculcating 
reciuisite values, skills and knowledge in aspiring teachers; and of 
ensuring, insofar as it is possible to do so, that gi-aduates reflect those 
learnings in their teaching. Further, teacher educators must com- 
pete with other career fields for candidates who bring to their edu- 
cation personal qualities that combine with rigorous preparation 
programs to produce competent teachers. 

Teacher education is once again being criticized and challenged, 
both fVom within and outside higher education. While some of the 
criticisms are appropriate, some, as authors Nelli and Nutter point 
out, are based on inadequate or misinterpreted information. Clearly 
those who administer and instruct in progi-ams of teacher prepara- 
tion have a special responsibility to evaluate those programs, both 
in terms of the challenges being raised by others and in relation to 
their own objectives for these training efforts. 
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This monograph is intfiMKled as a guide to improving the evalu- 
ation of programs of initial preparation for teaching ami, ultimately; 
teacher preparation itself. The authors' model suggests application 
to a wide range of evaluation situations, providing many specific 
examples of each component. 

The Clearinghouse gratefully acknowledges the work of Elissa- 
beth Nelli, assistant dean of education at the University of Ken- 
tucky, and Norma Nutter, director of student personnel services at 
Ohio University's College of Education, in producing this mono- 
graph. Thanks also to the content reviewers for their contributions 
to the draft manuscript. 

George Denemark 
Chair, Adrisorff lionrd 
KlilC ChvriHyhoiise on 
Teacher Kdnvation 
Interim iJireeton Xatianal 
Cofnieit for Acrreditation of 
Teacher tuilHeatioit 
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INTRODUCTION: 
THE SYLLOGISM 



A s teacher educators, our basic premise, our raison d'etre, is 
/% that we should make a positive difference in our students, 
who then, as teachers and because of that difference, make 
«JL J^iia positive difference in their pupils. In undertaking to 
evaluate teacher education programs, we evaluate whether and to 
what extent this premise is true. 

In actual practice, this apparently simple idea becomes enmeshed 
in conflicting conceptions of terms and operational defmitions. On one 
hand, teacher educators are engaged in an inherently conservative 
activity. We literally conserve accumulated knowledge and values 
about teaching, even to the point of ensuring that progress comes in 
ways consonant with the past. On the other hand, we subscribe to 
an inherently liberal proposition — that, through our actions, we can 
improve our students and their future pupils. Over the past decades, 
social forces have wrenched us from one extreme philosophy to another. 

We pursue our work within a seemingly intractable network of 
social, political, and economic institutions at the national, state, and 
local levels. We serve multiple clients, including future teachers, school 
administrators, parents, employers, pupils, legislators, taxpayers, 
and the teaching profession. And, finally, our graduates, whose si^c- 
cess is a measure of our success, serve in schools that are beyond our 
influence in crucial ways. 

For clarity, let us reduce the basic premise of teacher prepara- 
tion to a series of related statements: 

X is an effective teacher behavior, skill, attitude, or knowledge, i.e. , it has 
certain desirable effects on pupils. 

Through our teacher preparation programs, we teach future teachers to 
do, believe, or know X. 
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Thvivtbre. our i'utiuv toachors bucome oftootivu teachers, i.e., they pro- 
Uuco thust* (losirablo oftbcls in their pupils. 

This syllojfism is admittedly a ni'oss oversimplification; but, 
regardless of its loj4'i(?al validity or factual accuracy, this line of rea- 
soning has great psychological validity for teacher educators. More 
importantly, within this syllogism lie the niigor problems in concep- 
tualizing and accomplishing the task of preparing teachers. Let us 
examine the components of the tiyllogism to uncover some of these 
problems. 

X is ail effectivi» teacher behavior, skill, attitude, or knowledge, i.e. , it has 
certain desirable effects ou pupils. 

But w hat is X? Teacher preparation progi'ams differ in wh'U they 
propose to teach, the basis for selecting what is taught, and the basis 
for believing that what is taught will produce desirable effects on 
future pupils. 

Too often, what teacher educators believe to be effective or 
desirable teacher behaviors, skills, attitudes, and knowledge; the 
grounds on which they believe them to be effective; and what the 
positive effects on pupils are supposed to be are never made explicit 
in teacher preparation programs. 

We derive our ideas about what to teach future teachers from a 
number of sources: logic, our own experience as teachers, teacher 
folklore and teacher education folklore, research, the advice of experts, 
recommendations from practitioners, fads, and specific philosophies 
and theories. Everything we teach — from unit planning to educa- 
tional philosophy, from child development to eye contact — is sup- 
posed to make **better" teachers. 

The research base for teacher preparation is in a state of healthy 
infancy and certainly is far stronger than what we incoi*porate in most 
preparation progi'ams. However, even when we do have research to 
guide us, we find its usefulness limited. Knowledge about teaching, 
like most knowledge rooted in the social and behavioral sciences, is 
probabilistic and always subject to the variability of social context 
and individuals (Cronbach and Snow 1977). We always will be forced 
to draw on other sources and our professional judgment to determine 
curricula. Our most serious problem may not be lack of a research 
base for what we teach but the fact that we often cannot articulate 
exactly what it is we teach and why. 

We teach our future teachers to do X. 

But do we teach them? How much of the intended curriculum 
really is taught? How much do prospective teachers really learn, and 
how thoroughly? Teacher educators are getting better at assessing 
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students' active competence but still depend heavily on what Post- 
man and Weingartner ( UH)9) called the ** Vaccination Theory of Edu- 
cation" — simply exposing students to knowledge and hoping that it 
takes. 

One major problem is the longstanding lack of selectivity 
regarding whom teacher educators admit to and graduate from pro- 
grams. In many cases, it appears that teacher eciucators take almost 
all comers, give them high gi'ades, and gi'aduate almost all who endure 
(Brubaker 1976; Kapel 1980; Nutter 1983). Either teacher educators 
do a superb job of teaching X to students of widely varying abilities, 
including some poor or unmotivated ones; or else, while they may teach 
it to some, they set their standards so low that they actually gradu- 
ate some who have not learned it. Even when teacher preparation takies 
a competency-based or mastery approach, the criterion often becomes 
"the level of achievement that can be obtained by 98 percent of the 
studentfc? within one three-credit course," Teacher preparation pro- 
gi'ams differ in how well they teach to all students what they propose 
to teach and in how well they assess mastery. Few institutions can 
claim to do an outstanding job. 

Our future teachers become effective teachers, i.e., they produce those 
desirable effects in their pupils. 

Or do they? Even if teacher educators assume that an individual 
gi'aduate has mastered an adequate repertoire of generally effective 
teaching behaviors, there is no guarantee that the graduate will use 
those behaviors in the classroom, that the context will permit those 
behaviors, or that those particular behaviors will be appropriate or 
sufficient for the context. 

The public now appears to want to hold teacher preparation 
institutions responsible for teachers' successes and failures, some- 
times with narrow definitions of sncceiis *dr\d failure. Teacher edu- 
cators must assess their graduates* performance in the context of the 
types of schools and children they serve, evaluate the programs that 
prepared those graduates to teach, and try to maximize the pro- 
grams' success. 
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II 



CONTEMPORARY 
APPROACHES TO 
EVALUATING 
TEACHER 
EDUCATION 
PROGRAMS 



Contemporary approaches to evaluating' teacher prepara- 
tion pro^yrams are, at best, fragmented. Typically, pro- 
gram evaluation is sporadic, focused on some nan'ovv aspect 
of a particular program, and relatively informal. However, 
as Adams and Craig (1983) noted, some attempt at program evalu- 
ation is more common than the literature indicates. Teacher educa- 
tors do recognize the importance of evaluation and are constantly 
evaluating, at least informally, their courses and course components, 
student performance, the reactions of teachers to field-experience 
students — in sum, whatever concerns their daily work. 

The most common formal evaluations are those conducted by 
external agencies such as the National Council for the Accreditation 
of Teacher Education (NCATE) and state departments of education 
for initial or continuing program approval. These formal external 
evaluations tend to be highly descriptive and to focus on comparing 
documents (syllabi, course and progi^am objectives, catalogs, reports 
prepared by the institution) with standards for such things as spe- 
cific courses, credit houis, areas of content, and faculty qualifica- 
tions. As a result of time constraints, accreditation evaluations are 
fundamentally evaluations of programs as they exist on paper, with 
some attempt to verify the wintten descriptions during the brief time 
a visiting team is on campus. 

State approval procedures in particular have not been known 
for rigor, for attention to quality, or for finding many teacher educa- 
tion programs deficient. In recent years, some state departments of 
education have taken a more aggressive stance in using their evalu- 
ative authority to promote effective teacher education, but their pro- 
cedures remain entangled in state politics. 
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These external evaluations are valuable, even necessary; and their 
criteiia and procedures for conducting evaluations are generally solid, 
defensible, and reasonable under the circumstances. However, 
accreditation evaluations have clearly defined and limited purposes 
and cannot supply all the feedback teacher educators need. 

Increasingly, states are evaluating teacher preparation gradu- 
ates' competence, typically through standardized, paper-and-pencil 
tests of basic skills; and secondarily, these slates may use test results 
to evaluate teacher preparation progi'ams. For example, there have 
been some attempts to withdraw state approval of programs whose 
gi^aduates have low passing rates (^'Florida . . 1983). However, while 
teache)*s must have basic skills, evaluative decisions about teachers 
and eventually about their training progi*ams should not be made only 
on the basis of test scores, the relevance and significance of which 
are not yet cle'dv. 

The most common approach to internal program evaluation is 
the questionnaire completed by students at graduation or within the 
following year (Adams and Craig 1981, 1983). Such questionnaires 
indicate graduates' opinions of various aspects of their preparation 
programs and certainly yield valuable insights. By themselves, how- 
ever, they provide insufficient data for progi^am evaluation. Ques- 
tions left unanswered by follow-up questionnaires include: To what 
extent are the graduates' opinions influenced by the socialization 
process they undergo in the schools (Gubser 1981)? To what extent 
are they affected by beginning teachers' short-term survival needs? 
To what extent do they reflect universal stresses and concerns of 
teaching, regardless of the quality and extent of the teacher prepa- 
ration programs? To what extent do graduates take the preparation 
program's areas of success for granted and so do not identify these 
areas as high in importance? 

Simple managerial models (focusing on formal administrative 
relationships) and systems-analysis models are sometimes used as 
the sole approach to evaluating teacher preparation programs. How- 
ever, such iDUsiness-oriented models assume that accountability flows 
upward through a hierarchy of workers to middle management to 
upper management to directors and stockholders. Businesses also 
possess some clear, concrete criteria for success in terms of produc* 
tivity, cost/benefit, and profit. Outcome-oriented models, such as a 
behavioral objectives model, assume that only the product is impor- 
tant. 

Teacher educators operate in a distinctly different milieu, with 
both upward and downward accountability. The institutional hier- 
archy is neither the sole nor, in terms of social mission, the primary 
focus of teacher educators' accountability Teacher preparation dif- 
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ters radically from most busin(?s«e« in that the "product" is also the 
most proximate, tuition-paying client. 

Logic suggests that program evaluation should be an ongoing, 
comprf'hensive process within the teacher education institution 
(St. ■ -^eam 1982; Dillon and Stavkman 1981; Hord 1979). Evalua- 
tion should provide "an extensive record of the program that was 
actually implemented, how it compared to what was intended, a full 
account of the costs incurred in carrying it out, and, overall, how 
observers and participants judge the quality of the effort" (Stuftle- 
beam 1982, p. 154). In other words, understanding the process of 
teacher preparation is as important as evaluating the product, and 
all clients can give useful insights. 
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AN OVERVIEW 
OF A NEW MODEL OF 
EVALUATION AND 
PREREQUISITES 
TO EVALUATION 



propose a model for evaluating teacher education 
% ^ / programs based on a model developed by Robert E. 
%/%/ Stake (1967) for evaluating curricular programs in 
V public schools. We have adapted Stake's model with some 
modification of terminology and some elaboration of structure to fit 
teacher preparation progi'ams. 

Stake's approach is primarily transactional, providing the broad 
evaluative perspective appropriate for the complexity of teacher 
preparation progi'ams. A transactional model requires participa- 
tion by a wide range of concerned parties, not just "management" 
(House 1978), and so is more appropriate for teacher educators, who, 
given their multiple accountability and social mission, cannot prac- 
tically or ethically operate from a simple management pei spective. 
Other types of models are suitable for evaluations of limited aspects 
of programs and may be used within an encompassing transactional 

model. . 

The model presented here is intended to help organize thinking 
about evaluation and to serve as a model for designing an effective 
evaluation process within a given context. It is not intended to pre- 
scribe a lockstep approach to the actual process, because such a pre- 
sciiption would not be generalizable to all teacher education programs 
and to all evaluation situations. 

In presenting thi . inodel, we address teacher educators or 
administrators with no special expertise in evaluation per se, but with 
a general knowledge of issues related to teacher education and 
extensive knowledge derived from their teacher education pro- 
grams. In other words, we address knowledgeable educators evalu- 
ating either their programs or others* programs and needing a model 



and specific examples to jfiiide the evaluation process. We do not believe 
that teacher educators need evaluation experts to conduct evalua- 
tions that, while not the ultimate in sophistication and rigor, none- 
theless yield useftil information. Besides, teacher educators often have 
access to evaluation experts within the teacher education unit, the 
institution, or externally who can assist in implementing the model. 

"Evalm't<)r" and "teacher education faculty" are used as sepa- 
rate terms, although teacher educators will often be evaluating their 
own progi'ams. The "evaluator" may well be a team of evaluators, 
composed of individuals with complementary knowledge, strengths, 
and perspectives. 



ive major issues must be settled before the evaluation process 



begins. These are purpoHe, o ndience, qneationii, em luato)\ and 
Purpose 

The first, essential, and often neglected step in conducting an eval- 
uation is to determine its purpose. The purpose affects the types of 
data to be collected; the standards by which the program is evalu- 
ated; and. eventually, the types of action resulting ft-om the evalua- 
tion. Basic distinctions of purpose include ./o>v/<«^/<;e (evaluation to 
improve the program while it is functioning or being implemented) 
versus sum motive (an assessment, of the overall effectiveness of a 
program and its results) (Scriven 1967); and proavtive (evaluation to 
serve decision making) versus retroactive (evaluation to serve 
accountability) (Stufflebeam 1972). Purposes may be concealed; for 
example, an evaluation expected to result in the elimination of pro- 
gi'ams may be presented officially as a simple evaluation of status or 
a descriptive study Evaluations may alsc be conducted for psycho- 
logical or sociopolitical reasons, for example, to promote good public 
relations, motivate evaluees, cast blame and praise, or clarify lines 
of authority (i.e., who may evaluate whom^ (Nevo 1980). The most 
common official purposes are to improve existing and developing 
progi'ams and to eliminate weak ones. Smith (1977) discussed in detail 
the influence of political factors, social movements, and organiza- 
tional influences on evaluation studies in education. 

Audience 

Defining pui-pose includes defining the audience or clients to be served. 
For teacher educators, the definition usually includes multiple audi- 
ences. Audience refers to '7/<-e specific, relevant decision makers and 
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inforHfatiinf Hst^rs (not va^ue, passive audiencos) who will u:.o tlu? 
information that thu evaluation produces'* (Patton IJKSt), p. ot)). Vvom 
the pwspective of teacher preparation projiranis, the clients or 
audience for a particular evaluation could include teacher education 
faculty and administrators, colU^^iate faculty and administrators, 
school administrators, state personnel, the public, teachers, stu- 
dents enrolled in the pr(^p*am, and others, 

Questions 

Patton (U)7S) noted that merely identifying' the audience is not suf- 
ficient. Bivaluation planners must also identify precisely what it is that 
specific groups want and need to learn from the evaluation, Deter- 
mininji* the purpose of an evaluation also includes determininj; what 
questions the eventual clients need answered, which in turn deter- 
mines the types of data to be collected and with what decree of rij^or 
and credibility. 

Evaluator 

The purpose of an evaluation determines who will comhict the eval- 
uation, with the mtijor options being an internal evaluator or an 
external one. 

Internal evaluators, people who have some deg'ree of connection 
to a projfi'tini and who perhaps developed it and work in it, offer sijy- 
nificant advantages — familianty with the program, its purposes, its 
constraints, and its faculty — and often have strcmg motivation to 
complete the evaluation in order to improve the program. Their dis- 
advantages may include personal bias because of their stake? in the 
evaluation's outcome, shortsightedness ft'om overfamilianty with the 
program, and possibly a lack of comparative experience with other 
programs. 

External evaluators, those with some degree of distance from 
the progi*am, offer advantages of relative objectivity (although they 
may be influenced by human relations), a fresh perspective, and 
usually some comparative experience. Explaining a program to an 
external evaluator may be enlightening in itself, and various audi- 
ences may regard an external evaluator's judgments as more trust- 
worthy. On the negative side, external evaluators mtiy bring to an 
evaluation preconceived notions, ignorance of the specific situation, 
and inadequate time and resources to ctmsoUdate and consider the 
relevant information. 

A common compromise is to use both external and internal eval- 
uators* By determining beforehand what questions should be 
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answered, the evaluation planners can select an evaluator or team of 
evaluators with appropriate skills and perspectives. 

Utilization 

Utilization, the final dissemination of the evaluation report and the 
types of decisions to be made based on the results, must be planned 
ft'om the beginning. Often, evaluation results appear to go unheeded. 
Several steps can maximize the chances that evaluation results will 
make an impact. 

First, all the various audiences must be involved in the evalua- 
tion process from the beginning. Second, an active communication 
system must present them with emerging evaluation data to review 
and interpret; and, third, the evaluation must arise ft'om their spe- 
cific information needs (Craig and Adams 1981). Finally, the evalu- 
ation report(s) must be written in language comprehensible to the 
various audiences, the data must be understandable, and the total 
report must be written so that practical applications can be drawn 
fVomit (Cooper 1983). Campbell (1979) recommended incorporating 
formal procedures that permit all interested parties to make addi- 
tions and corrections to the official evaluation report. 

Plans should also include a method for recording the evaluation 
process itself and monitoring whether external events affect it (Fil- 
stead 1979). The final report should enable readei^s to judge the depth, 
breadth, and integrity of the evaluation process. 

Patton (1978), in a study of the impact of evaluation on federal 
programs, found that, aside from political factors, the most impor- 
tant factor in whether results were used was persoml in volvetmnt'. 

the presence of an identifiable indiv dual or group of people who person- 
ally cared about the evaluation and he information it generated. When 
such a person or group was present, evaluations were used; where the 
personal factor was absent, there was a correspondingly marked absence 
of evaluation impact, (p. 64) 

He also found that evaluation — which is only one factor in ongoing 
program development— often has quiet impact by reducing uncer- 
tainty for decision making, promoting gradual change, and acceler- 
ating change that has already begun (pp. 31-34). 

Planning for utilization from the very beginning, involving con- 
cerned parties in the evaluation processes, and deliberately guiding 
the evaluation toward usable results maximize the chances that those 
results will be heeded and that concrete actions (program revision, 
elimination, or enhancement) will occur based on them. 
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HKLI* WITH PUKLIMINAKY 
l»LANNINC; 



Staki^ ( U)7(i, pp. listed 12 (lucstions intcndi'd U) clarity the 
aurt'ement betwoon an external evaluator and the eontractinK 
parties. This checklist serves as a ii.s(»t'ul orietUatlon i^mlv tor any 
('valuation effort. Questions ran^'e from "Who wants an evaluation 
study?" to "What resources are available for the conduct of this study?" 
to "What would be a suitable plan for reporting' tin? tindiuM's?" 

Smith (1982) discussed in detail the preliminary steps in plan- 
ninfi' an evaluation: detininu' scope, purpose, (juestions, methods, and 
design. Hoth's workbook (UKSl) provides assistance with planning", 
problem identitication, and basic concepts of instrumentation and 
statistical aiuilysis for evaluation. 
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IV 



THE PARTS 
OF THE MODEL 



Figure 1 displays a schematic of the proposed model for evalu- 
ating teacher education programs. The parts, their interpre- 
tation, and their interrelations are as follows. 

RATIONALE 

ationale refers to the overriding philosophy and basic pur- 
poses of the teacher education program. Frequently, the 
rationale is implicit rather than stated. 

Whether implicit or explicit, the rationale requires examina- 
tion. Where it is implicit, it must be made explicit. Where it is explicit, 
it must not be taken at face value but examined for coherence with 
all other aspects of the program. Otherwise, the evaluator will make 
judgments about whether a program accomplishes its purposes 
without knowing the broad basis for those goals. Official statements 
of guiding philosophy tend to be vague, all inclusive, and lofty The 
evaluator has the task of eliciting more concrete details from the 
owners and clients of the program: 

Who are the proximate and ultimate groups affected by this program? 
How is it intended to affect them? 

What types of settings are the gi-aduates to be prepared for and to serw 
what types of students? 

What specific concept of the teacher's role guides the program? 

Even the answers to these questions are likely to be vague at 
first; the evaluator must probe until fairly concrete answers are 
obtained. 

The importance of the rationale is, first, that it provides a basis 
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Figure 1: Schematic of a Proposed Model for Evaluating 
Teacher Education Programs 
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for seleotinj? rcfm-nco tyroups that will pass ju(li?ment on various 
aspects of the teacher eclucation projiji-am and for selecting the spe- 
cific standards by which the program should be judged. Second, the 
rationale provides the basis forjudging whether all parts of the pro- 
gi'am — intentions, cuiriculum, and outcome — form a coherent whole. 
The rationale itself is subject to judgment and should be scrutinized 
closely; but, once accepted as legitimate and worthwhile, it strongly 
influences the bases on which the program may be found commend- 
able or lacking. 

Before considering the cells of the matrix shown in Figure 1, we 
will briefly define each category. 

THE MATRIX 

Intentions 

Intentions refer to what teacher educators variously call goals, 
objectives, or intended outcomes, however they choose to formulate 
them. In other words, intentions are those aspects of the teacher 
education progi'am that are deliberately planned. 

Teacher educators generally recognize the importance of mak- 
ing some of their intentions explicit; for example, teacher educators 
usually \snite objectives. The weaknesses have been in specifying other 
types of intentions, linking intentions specifically to curricula, and 
then linking intentions to actual outcomes of progi'ams. 

Frequently, intentions are vague and unspecified for a given 
teacher preparation progi'am. Teacher preparation cumcula may arise 
from tradition, historic accretion, imitation of other programs, 
piecemeal responses to external standards (e.g., state minimum 
requirements for certification), fads, or the interests of key faculty, 
with little or no attempt to consider intentions independent of exist- 
ing curricula. It is difficult to focus first and only on what a teacher 
education program should be. 

Data 

Data are those descriptive facts evaluators collect by whatever means 
to assess whether program intentions are in fact realized and whether 
significant, unintended events occur. 

Formal benchmark data for teacher education do not exist in most 
cases, and comparison across institutions is often difficult or politi- 
cally impossible. Where data do exist or can be collected, they tend 
to relate to limited, albeit important, aspects such as enrollments, 
credit hours, salaries, budget, and staffing ratios, but generally with 
no uniformity in compiling and reporting the data. Peseau and col- 
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leagues (0.^',, IVsoau and Orr 1979, 1980; Roseau 1982) have* beK'un 
to compilo and analy/.o benchmark data on teacher education (staff- 
ing', budget, etc.), but at present little else is available. To a great 
extent, comparative data must 'derive from the evaluator's own 
experience — personal familiarity with both the history and current 
status of teacher education, similar programs and contexts, and the- 
ory and research in teachc»r education. 

Given the general lack of standard categories, the evaluator 
necessarily makes subjective decisions about which data to collect 
and review. The evaluator shouUl be guided by the previously devel- 
oped elements: the information needs of both clients and owners of 
the program, the evaluation ({uestions, and the rationale and intents 
specified by the teacher educati(m faculty. While the evaluator must 
collect a wide vanety of data, he or she cannot collect all possible data. 
The priorities of the evaluation effort, combined with practical con- 
straints, essentially (letermine (lata selection (Nevo 1980). 

The word data is sometimes taken to refer only to cjuantitative 
information or even only to the results of experimentation. However, 
the complex nature of teacher preparation programs demands that 
both quantitative and (|ualitative da*a be used to complement each 
other and produce a broad enough picture. 

Most teacher preparation programs are not suitable for large- 
scale experimentation, although small areas are subject to empiri- 
cal, quantitative analysis. Experimentation requires limiting and 
tightly controlling variables, whereas the task of teacher educators 
is "to manipulate as many variables as necessary to achieve multiple 
objectives which are often in conflict and of varying importance" (lanni 
and Orr 1978, p. 90). As Cooper ( 1988) noted, the data and data anal- 
yses most useful for internal decision making are not necessanly those 
suitable for, or emanating from, empirical research. ( -ompared lO 
researchers, decision makers are less concerned about validity and 
reliability and more concerned about having data with direct impli- 
cations for their own context. 

Qualitative data — that is, 

defailed (icsrripthnis of situation.s, oviMits, people. iiUcrat'tions, ami 
observed hehaviors; dnrrf (iHotatitufs from people about thcii' i»xperi- 
ences. attitudes, beliefs, ami thoufjhts: ami excerpts oi* entire passajjes 
from documents, correspondence, records, am! case histories . . . data 
collected as open-ended narrative without attempting: to fit profjram 
activities or peoples' experiences into pi'cdetermined. standardised cat- 
egories (Patton 1980. p. 22) 

— can yield a rich understanding of the actual process by which 
teachers are prepared, and are useful for generating theories (Rei- 
chardt and Cook 1979, p. 17). Quantitative data, on the other hand. 
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may support or contradict (|ualitativo data and aro useful for vevi- 
fyinia; tluunnt^s, Thv two approaches serve as checks on each other. 
Contradictions between the data yielded by each approach often lead 
to valuable insij^'hts and closer examination of problematic situa- 
tions. Quite simply, the evalualor practices "trian^Uation" — the use 
of multiple sources of data {including multiple ol)servi»rs) and mul- 
tiph» methods to zero in on the state of things, as the inherent biases 
in the differinia: sources and methods offset each other (Den;5in 1978). 
Keichardt a. id Cook (1979) liave noted that a combination of quali- 
tativi* and ([Uantilatiw methods is espticially approprii^te for evalu- 
ations serving' multiple purposes^ 

One (iualitati\'e techniciue that can be illuminatins* but has rarely 
hvvn used in an evaluation of a teacher preparation program is par- 
ticipant observation. With tills ap|ir-'^-^^ the (?valuator enters the 
program to whatever extent, for whaitHvr length of time, and in 
whatiA'er role best suits the evaluation's purposes and context. The 
evaluator obserws what actually happens, participates in it, and uses 
guided observations and interview's to assess the actual process of 
the program (DeriJ'-in 1978). 

Contrary to current practice in many teacher pre|)aration pro- 
grams, collecting outcome data does not in itself constitute program 
evaluation. For example, facts sucli as how many graduates of a teacher 
prt»paration program pass a competency test, how wc?ll they score as 
a group, how many are employed, how W(»ll they are rated by their 
sup(»rvisors, and how well their pupils achieve s^v nothing about the 
process of the teacher e(h;cation program itself. Many critical 
unknowns remain. For example, what was the caliber of the stu- 
dents originally attracted to the program? What changes did the 
teacher education program produce in its students? What are the 
contexts within which grmUiates teach and the effects of those con- 
texts on [hviv tiuiching? And what are the caliber and types of pupils 
thi\v teach? 



Anti^cedents 

An antecediuU is "any condition existing prior to teaching and learn- 
ing which may relate to outcomes'' (Stake 19(57, p. Antecedents 
necessarily limit an(t shape what is done in a given program and 
therefore affect outcomes. 

Anteceder.ts for teacher preparation programs include, but are 
not neces.^arily limited to: characteristics of students, faculty, and 
physical and fiscal rt»sources: institutional policies, governance, and 
limitations; ihv program's history, tradition, and geogi^aphic loca- 
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tion; aiul state? mandates. Budget is an ospuclally persistent delim- 
iter ()f'proii»TanKsc();)c» and effectiveness. 

Interactions 

Interactions constitute the actual curriculum in action, that is, the 
interactions between students and faculty, students and field per- 
sonn 1, students and facilities, students and students, and students 
and knowledge. A curriculum is a dynamic process of events and 
experiences. Yet. evaluators of teacher education programs have 
generally focused on the paper curriculum — the courses listed in a 
catalog, lists of objectives, syllabi, and program guidesheets. They 
have taken the paper curriculum at face value and evaluated it from 
a narrow, (luantitative perspective, which may be summarized in a 
question: "Is 'X' content covered somewhere?" 

In recent years, devotion to systems analysis and behavioral 
objectives, with their emphasis on outcomes and, to a lesser extent, 
on Inputs, has led some teacher educators to give less attention to 
actual educational processes. However, focusing only on goals and 
outcomes, one has no idea of the middle term (the process) and whether 
or how it affects outcomes. Moreover, discrepancies between the paper 
curriculum and the acrtual curriculum occur routinely. For example, 
cunicula are strongly altered by personnel factors — changes in staff, 
part-timers, variations from term to term in staffing particular 
components, degree of faculty commitment to the official, intended 
curriculum, orientation of new faculty, and the degree to which 
implementation of the curriculum is monitored by the faculty. Devia- 
tions are to be expected and must be asseswsed in program evalua- 
tion, 

8tarkman et al. ( 1979) provided an example of a design for col- 
lecting evaluation data that appears sound, except that it focuses on 
student-related antecedents and outcomes to the exclusion of pro- 
cess. Without infoi-mation on both process and outcomes, identify- 
ing areas of improvement is difficult. 

Where outcomes are evaluated without knowledge of implementation, the 
results seldom provide a direction for action because the deoivsion maker 
lacks information about what produced the observed outcomes (or lack of 
outcomes). This is the *'black bo.\" approach to evaluation. (Patton 1980, 
p. m 

P'urthermore, programs may be found deficient and even elimi- 
nated on the basis of outcome data when in fact the intended pro- 
gi'am (the interactions that should have occurred) has never been 
adequately implemented. 

As Galluzjso ( 1982) noted, gaining a description of the processes 
of a teacher education program and thereby an understanding of how 
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it actually works is in itself an important jfoal for an,v program eval- 
uation. Similarly, Hall(lHSl) assurtcul that follow-up studies of grad- 
uates and studies that focus only on one component are not the jioal 
of program evaluation. Teacher educators need a picture of the total 
continuum anil the interrelationships of the parts of their programs, 
Towai'd this end, Hall recommended studies assessing the effects of 
basic program components. 

Outcomes 

Outcomes are those effects attributable to the program; those 
occurrences, whetlier intended or unintended and whether positive 
or negative, that manifestly result from the program. Outcomes are 
short- and long-range, cognitive and affective, individual and group. 
They can be assessed throughout the cuniculum as well as at its end. 
Some types of outcome data will not be available to evaluators unless 
there has been a system for collecting them over time, 

Recently, teacher eciucators have given considerable attention to 
assessing outcomes for graduates through various forms of compe- 
tency assessment, scores on standardized achievement tests, scores 
on tests of basic skills and professional knowledge, follow-up studies, 
and studies of pupils' achievement, Weber and Cooper (1978) offered 
an example of a progi'am evaluation based on a "goal attainment" 
perspective, in which outcomes are assessed in the light of inten- 
tions. 

Teacher educators have encountered difficulty in sptrcifying 
desirable outcomes for theiv graduates and, even when desirable 
outcomes have been specified, in assessing those outcomes. Although 
much more needs to be done, current efforts in design, instrumen- 
tation, and use of follow-up studies ^e.g. . Adams 1978; Arends 1978; 
Ayers 1978) represent commendable efforts to actjuire an important 
source of data, heretofore almost universally missing, for program 
evaluation and program improvement. Such efforts can be invaluable 
reseaix'h in themselves (Schalock 1979). VaHous types of outcome data 
that are collected include terminal assessments of gi^aduates' com- 
petence, employment information about gi'aduates, graduates' opin- 
ions of their preparation programs, administrators' ratings of 
gi'aduates' performance, and, occasionally^ observational and pupil- 
achievement data. 

Often, "program evaluation" is not clearly distinguished from 
"follow-up and evaluation of gi^aduates" (e.g. , Middleton and Cohen 
1979; Hord and Hall 1978; Hord, Savage, and Bethel 1982), Simply 
ac(iuiring follow-up data does not constitute progi^am evaluation. 
Rather, follow-up data must be integrated into the overall picture of 
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how these results came to be and their relation to the teacher edu- 
cation program itself (Gubser 1981). How a teacher education pro- 
gi'am produces its effects is as important as, and in some instances 
more important than, the typical finding that it produces teachers 
regarded as adequate or slightly bet' er than adequate. 

One common approach to follow-up is to send questionnaires 
eliciting the graduates' opinions of.their preparation program. The 
results are taken at face value as a reliable evaluation of the pro- 
gram, which then should be the basis for program revision and 
improvement, usually by adding something (a course in classroom 
management, more field experience) to the cuniculum. Gubser (1981) 
suggested that these questionnaires may tell teacher educators what 
"common wisdom tells graduates to think they acquired, or did not 
acquire. Surveys tell us more about peer knowledge and group 
methodology than th^ do about preparation programs completed by 
prospective educators" (p. 34). 

The isolation of most follow-up studies from description and 
evaluation of the actual curriculum, which is the reality in which fac- 
ulty are immersed, may account for the common report that faculty 
do not incorporate the results into program development (e.g, , San- 
defur 1982). 

RathF ..;ited in Gubser 1981, p. 31) stated that program evalu- 
ation shoi. > also not be confused with only assessment of individual 
teaching sKills or with assessment of the competence of individual 
gi'aduates. Sandefur(1970, p. 12) recommended a broad approach to 
gathering follow-up data in four categories — career-line data; direct 
classroom observation; pupil, peer, and supervisory evaluations; and 
standardized measures. 

Current thought in process/product research stresses evalua- 
tion based on pupil outcomes (e.g., Peck and Tucker 1973). This 
approach is theoretically and philosophically consonant with the 
teacher education syllogism presented in our first chapter. Unfor- 
tunately, the current state of the art of evaluation and research does 
not permit pupil outcomes as a practical basis for evaluating pro- 
grams because we cannot yet tie pupil outcomes to specific program 
contents. Rather, product research can serve as a guiding perspec- 
tive for evaluation of teacher education programs. 

Contingencies and congruences 

Contingencies and congruences are aspects of the context of a pro- 
gi-am. Contingencies are those external factors that impinge on pro- 
grams, such as a hiring freeze that obliterates programmatic 
expectations for mandated content in a new area; congruence refers 
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to tho (lyjfi't'tf of I'ori't'spondonce btaweon the iiitundtul and the actual 
projrrams. such as a foursc intomUul to provide conttmt in tht? social 
foundations of education that a faculty member orients toward cur- 
riculum theory. 

Part of the evaluator's task is to assess to what extent the pro- 
gram achieves the expecftations of the relevant groups (congruence?) 
and whether the vanous components of the pro^Tam should be chang-ed 
to reflect changing conditions (contin^'encies) (Stufttebeam et al. 1971). 
In the examples just given, attainment of congi'uence would recjuire 
that the progi-am faculty either alter course expectations, or demand 
that expected course ccmtent in social foundations be, in fact, pro- 
vided. To respond to the contingency situation imposed by the hiring 
ft-eeze, faculty might refuse to provide the mandated content, appeal 
the freeze, retrain themselves in the needed area, or "borrow" 
instructors from related units. 

Standards 

Standards are those rules or guidelines against which the quality or 
value of the teacher education program is measured. We have modi- 
fied Stake's original model, which showed standards as part of the 
judgment component. We prefer to think of standards as a screen 
through which data relative to the intended antecedents, interac- 
tions, and outcomes of a teacher education progi'am are filtered to 
produce, eventually, judgments about parts of the program and the 
program overall. 

One of the evaluator's most difficult tasks is selecting the eval- 
uation standards, determining which gi'oups' standards are rele- 
vant, and determining what weight particular standards should 
receive in judgments. A multiplicity of possible standards — too many 
for practical use — is normal, and so the evaluator must choose. 

One category of standards includes those explicit, formal stan- 
dards prepared by external groups such as NCATE, state depart- 
ments of education, and professional and learned societies. In addition, 
institutions of higher education and departments within those insti- 
tutions may have their own standards. These standards represent a 
group's consensus on what a program should be and, for practical 
purposes, what it must be either for continued existence or for 
professional recognition. 

P'ormal external groups are one valid source of standards, but 
formal external standards alone do not suffice to evaluate fully all 
relevant aspects of a complex teacher preparation program. Often, 
external standards have been written broadly (vaguely, some would 
argue) to cover a wide range of acceptable variations. Thus, the eval- 
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uator on site must study the situation and apply more specific cri- 
teria from experience. 

A second type of standards arises from the goal-attainment 
perspective (Provus 1971), in which programs are evaluated solely 
on whether they achieve the goals specified by the program owners. 
Certainly the clarity of a program's goals and the degi'ee to which 
they are achieved are relevant. However, goals themselves must be 
evaluated, and the process for attaining those goals studied. 

A third type of standard is the identified needs of clients (Joint 
Committee 1981; Pat ton 1978) and the extent to which those needs 
are met. As mentioned previously, teacher educators serve multiple 
clients, including administrators, state personnel, the publiCi teach- 
ers, and students enrolled in the program, 

Finally, there are informal or uncodified standards held by the 
various groups involved in or affected by teacher preparation — the 
teaching profession, the public, students and graduates of the pro- 
grams, school systems and administrators, subgroups of the public 
(parents, taxpayers), judges, media, and others. 

Drawing on standards from a variety of reference points, 
including past experience, helps the evaluator offset the relative lack 
of specific, generally accepted standards for teacher preparation 
programs. 

Nevo (1983) pointed out that the evaluator does not always have 
a choice of specific standards because the program owners may want 
specific standards supplied. The evaluator is responsible, however, 
for seeing that the choice of standards has a sound justification. 

Judgments 

Judgments are statements of the quality or value or effectiveness of 
a program and its components. Judgmen* may be a threatening con- 
cept to evaluees, and often, emphasis ^/laced officially on providing 
information for making decisions (e.g. , vJror <xQh 1963) or on improving 
a program (Cronbach et al. 1980; Stufflebeam 1982). Also, an emphasis 
on scientific method and on reporting only discreet, observable 
behaviors and events may cause evaluators to focus too strongly on 
description in the interest of objectivity, and thereby i elinquish their 
right and responsibility to make judgments about program quality. 

Judgments of merit, worth, quality and feasibility are inherent 
in the evaluation process. Both decision making and program 
improvement rest on judgments about the present situation. As 
Stufflebeam (1982) noted, some programs will be judged unworthy 
of the effort required to improve them. 

Depending on the purposes of the evaluation, judgments may 
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be expresst'd in absolute terms, such as "good," "adequate," or "pooif 
or in relative terms, such as "better than X program" or "needs 
improvement." Again depending on the purposes of the evaluation, 
evaluators may also provide judgments accompanied by recommen- 
dations for action. 

Judgments about teacher education prog)'ams tend to be vague, 
bland, hackneyed, and self-evident. Typically, they call for simply 
adding something to the program (Katz et al. 1981). Conscientious 
evaluators must strive for judgments that are rigorous, specific, and 
meaningful to evaluation users. 
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et us now consider each of the cells of the matrix shown in Fig- 
ure 1 in order to understand how the model actually works. 

INTENTIONS— ANTECEDENTS 



^ I he power of antecedent conditions is easily overlooked. Pro- 
.Am gram goals, arrived at through hours of discussion and research, 
nay be viewed as the only structure that determines progi'am con- 
tent and results. In real life» however, there are always constraints 
that shape what can be achieved. Evaluations of teacher education 
programs must recognize limitations, because recommendations 
resulting from evaluation will either be implemented in the context 
of the existing antecedents or will require modification of those 
antecedents. 

Evaluators must distinguish between stated intentions and actual 
practice. For example, institutions that claim to value equally the 
three areas of teaching, research, and service, but then reward fac- 
ulty for achievements in only one or two of these areas, create an 
incongruence between antecedent conditions and program expecta- 
tions. Regardless of the stated intentions of the program, anteced- 
ent conditions inexorably will control it. 

Antecedent conditions that influence the intended curriculum of 
a teacher education program fall into three major categories: stu- 
dent characteristics, faculty characteristics, and suppoil facilities and 
personnel. 
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student characteristics 



The largest number of students seeking entry into teacher educa- 
tion programs in the United States are eflirplled in public institu- 
tions of higher education, many of which have policies of open 
enrollment. Thus, the opportunity for selective admission at the 
ft*eshman level is often nil. Even colleges with selective admission face 
the problem of accjuinng a data base predictive of an applicant's future 
performance as an educator 

In most institutions, formal admission to a teacher education 
program occurs in the first or second year of a student's undergrad- 
uate course work. Although actual screening for teacher prepara- 
tion programs seldom occurs before entry into college, the professional 
programs can draw on information relating to previous general and 
subject-matter competence. Student self-reports, college-entrance 
tests, and aptitude/personality tests may provide pertinent infor- 
mation. Many entering students have completed college-entrance 
testing such as the American College Test (ACT) or the Scholastic 
Aptitude Test (SAT), which may identify academic. strengths and 
weaknesses. The ACT Student Profile Report details self-reported 
needs, interests, plans, and goals and may serve as a predictor of 
academic strengths and weaknesses relative to the selected institu- 
tion and program. 

Since the 1960s, almost all undergraduate teacher preparation 
programs have used some system of admission review; according to 
Brubaker and Patton (1975), the median number of selective items 
in operation at any one institution is six, and the most frequently nsed 
item is college grades. Other items are or could be used. For exam- 
ple, Arnold et al. (1977) recommended trial adoption of batteries of 
tests that best fit program goals and philosophy. An analysis of mul- 
tiple selection criteria in six states (Uno et al. 1981) indicated that 
smaller colleges and universities are more concerned than are larger 
ones with selection criteria and especially with personal evaluation 
procedures. 

The pool of applicants available to any program is likely to pos- 
sess certain characteristics in terms of ability geographical origin, 
attitudes, and socioeconomic status. The original pool and how the 
teacher preparation program selects from that pool form a powerful 
antecedent condition. 

A number of states have mandated testing programs prior to 
admission to teacher education programs. Where test results are used 
to screen applicants, the applicant pool available to program areas 
is modified according to the area of testing and the standards for 
screening. Directly or indirectly, the characteristics and abilities of 
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studonts in a program will infliuuico the content , speed, and level of 
instruction. 

Amodeo and Martin (U)82) described a sophomore-level diaj?- 
nostlc s(jreening process for teacher education students at New Mex- 
ico State University, with a prescreeninj? (orientation) component, a 
basic skills component, and an assessment of teaclung potential Such 
a procedure stronjjly affects who enters a program and results in a 
useful data base for evaluation. As long as the selection system is 
actually enforced and the critei'ia on which the system is based are 
valid, the matcli oi'congi'uence between intentions and data is exact. 
If the nature of the applicant pool changes, tiie change can be detected. 
And, if change is seen as necessary, programs can be modified on 
the basis of screening results. 

Kxample: Intentions— Antecedents (Students) 

Sample of wuluator's questions for identify i!)g program intentions and antcced- 
ont ooiKlitions regarding studeuts. 

• In the context of stated program goals, what characteristics should students 
in this program exhibit? 

• Do institutional and program policies encourage students with these char- 
acteristics to apply to this program? 

• Does the pool of candidates available to this progi'am exhibit these charac- 
teristics? 

• By what mi?a!is does the institution/progi'am screen out students lacking these 
chai'aeteristics? 

• What are the constraints on the program's ability to attract and hold desired 
students? 



Faculty characteristics 

Clark (1961) suggested that observations of faculty should be ori- 
ented toward evaluating the kinds of outcomes and objectives for which 
the institution strives, In order to function effectively, faculty must 
know "whether the institution values superior teaching, scholar- 
ship, productivity in writing and research, dedication to the insti- 
tution, a broad educational background, personal warmth, good 
personal habits, or a combination of these variables" (Arnold et al, 
1977, p. 74). Lack of overlap between institutional goals (or what the 
institution rewards with recognition, salary tenure, and promotion) 
and progi'am intentions (which faculty members might describe in 
terms of student achievement and school-district interactions, for 
example) creates a potential for conflict at the antecedent phase, before 
the program is even implemented. If the reward system favors fac- 
ulty who neglect teaching and service roles in order to publish, if it 

25 



33 



makes no distini'tion botwoen effective and ineffective teaching, or if 
it does not recoijnize faculty who stay productive and cun'ent in their 
fields, the quality of a program will suffer regardless of its stated 
goals. 

Institutional policies and history determine the experience, 
attitudes, and educational level of faculty, as well as their willing- 
ness to allocate time to program needs. The scholarly qualities asso- 
ciated with outstanding teachers in higher education (degrees 
completed, areas of specialization, research published, papers given, 
and awards received) are more readily measured than are personal 
qualities (agreeable nature, stability, enthusiasm, integiity, and high 
cultural attainment), although students report being affected at least 
Hs much by these personal qualities as by the scholarly ones (Isaac- 
son, McKeachie, and MilhoUand 1963). 

Example: IntentionH— AntecedentK (Faculty) 

Sample of evaluator's questions for identifying program intentions and anteced- 
ent conditions reKarding./a<.'»/<//. 

• What faculty characteristics does this institution traditionally reward with 
salary increases, promotion, recognition, ami tenure? 

• In the conte.xt of stated program goals, what characteristics should this pro- 
gram's faculty exhibit? 

• What progi-am demands will faculty he required to meet? 

• What recruitment and hiring policies does the institution use to attract and 
retain faculty with the desired characteristics? 

• How will faculty without desired characteristics be removed ft'om Involve- 
ment with this program or be given professional development opportunities? 

• Is there a correspondence between what the institution wants from faculty 
and what the program wants? 

Support facilities and personnel 

NCATE's emphasis on resources and facilities for the campus-based 
components of programs (Standards 5 and G-6) under.scores the need 
for "an environment which supports the basic teacher education pro- 
gram it offers" {NCATE Stmidards 1982, p. 25). Without sufficient 
funding and effective management, libraries, media centers, class- 
rooms, resource rooms, and clinics cannot contribute to program 
needs and goals. The availability of such resources is critical; so are 
their use and maintenance. For example, a program goal might be 
to train students to use computers for classroom instructional needs, 
yet if the campus computer facilities are scheduled so that little free 
practice time is available, or if they are continually closed for repair, 
these antecedent conditions will prevent achievement of the pro- 
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jfraniV inti'iitions. In similar ways, tundinji* policii^s and allocation for 
i»esouiTos provide ant(»i?tulent intentions that aftoet pvogvam implo- 
niuntation. Included in campus-support areas arc clerical and teach- 
ing assistancts data-managt^ment systems, communication systems, 
repair and maintenance systems, and lu'overnance systems. 

A related set of antecedents at the level of intentions is the insti- 
tutiim's willingness and ability to provide specialized resources. For 
example, if a program intends to train its students in audiovisual 
techniiiues, the institution must actjuire sufficient eijuipm^nt and 
place it where students can use \L A program to train prospective 
teachers in mainstreaming skills will require specific instructional 
materials for student use, faculty c(m\petent in providing the t)'ain- 
ing, and some provision for related field or clinical experiences. In 
the same way, an early-childhood-education program that is designed 
to provide on-site training, but lacks on-campus, early-childhood 
laboratory facilities, must have alternative arrangements for use of 
local day-care, nursery, or kindergarten eiMiters. Without such 
arrangements in place, the program will not attain its goals. 

Institutional climate — the extent to which the institution's 
administration and faculty outside education recognize and value 
training for teachers — constitutes a powertld antecedent condition 
tor all teacher educaticm programs. As Denemark noted (19S2), 

Two of the brijjhtcst spots on mi otherwise gloomy contemporary horizon 
in teacher education are the recent statenientsof key administrators tVom 
prestigious universities supportinjj the importance of hiji'her education's 
responsibilities for the improvement of elementai'y and secondary schools 
and the need for an effective school of inlucation to carry out those respon- 
sibilities, (p. W)) 

Denemark referred to Stanford University's President Donald Ken- 
nedy and University of California-Herkeley\s Chancellor Ira M. Hey- 
man. both of whom evinced support and respect for teacher 
preparation progi'ams and schools along with recognition of some of 
the problems unique to education faculty. Mo.^t teacher educators, 
however, serve in institutional climates that downgrade teacher 
preparation. In such sun'oundings, education faculty m^v be inclined 
to imitate the style and interests of their arts and .•sciences col- 
leagues, to the detriment of both teacher preparation and the public 
schools (Heyman. cited by Denemark 1982. p. HO). 

Schools and clinics for f eld experiences provide additional ante- 
cedent conditi(ms. The relationship between teacher preparaticm 
programs and public schools creates a context for future interac- 
tions. Stated and real intenticms may divei^ge, depending on the per- 
ceived self-interests of both parties. For example, a school principal 
who has found that teachei' prepai'ation students have enhanced 
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classroom activities and are valued by teachers is likely to facilitate 
future field experiences. A school or district administration that 
perceives education faculty as accessible and effective providers of 
needed research and inservice is likely to cooperate in projects for 
mutual benefit. However, when principals, teachers, and district 
administrators view field experiences as annoying and irrelevant 
intrusu)ns, access to schools is likely to be denied. In this case, the 
antecedents will limit the program's field capabilities regardless of 
what the progi^am intends. 

Intended antecedents also include the w^'s in which a program 
plans to capitalize on special resources or turn potential liabilities 
into assets. For example, a progi'am in a rural area might specialize 
in preparing teachers to meet the needs of rural schools; or a pro- 
gram in a metropolitan area might plan to supplement its resources 
with those available in the community. 

Kxample: Intentions— Antecedents (Support Facilities and PerHunnel) 

Sample of evaluator's qucvStions for identifying program intentions anil anteced* 
ont conditions a^garding {Support favH it ief< ami pevsoimeL 

• What facilities and cquipnietit are required tor achievement of progi-ani goals? 
What locations are most feasible? 

• What are the repair and maintenance procedures? 

• What scheduling of equipment is necessary to reali?.e program goals? 

• What is the history of institutional support for teacher education programs? 

• What is the current status of the prosram within the institution? 

• What is the history of relations between the progi'am and local school sys- 
tems or other field resources? 

• What is the ci»vrent status of the profjram within local school systems? 

• What field sites are necessary for students in the program? What sites are 
planned for? Are these plans realistic? 

Identifying all significant antecedents is not an easy job. As Smith 
(1980i p. 3) noted, **In the entangling network of federal, state, and 
local agencies and private and professional groups, the actual polit- 
ical and social forces shaping pedagogical education are no longer 
obvious." Yet these hidden forces have such power to control pro- 
grams that they should be identified so far as is possible. Yet pro- 
gi'am evaluation tends to neglect this potent area, perhaps because 
of the confusion and politics involved. 

At the intentions stage, the evaluator's job is to discriminate 
among the antecedents by separating the real from the paper inten- 
tions and to begin to discoN-er the limits of w^hat may be expected of 
the pvogi*am. Sometimes, deliberate attempts to improve progi'am 
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quality may croato conflicts with antececlent conditions. A pro- 
gram's faculty might decide to limit admissions to students with ACT 
composite scores of 22 or above as a move toward iiuality control. 
Faculty members miKht be unaware that the pool of available stu- 
dents exhibits an average composite score of 18. If intentions were 
to be implemented, the program would then admit very few stu- 
dents. Awareness of such a discrepancy between intentions and 
antecedent conditions would pemit consideration of alternatives: The 
pool of students might be altered through changes in recruitment, 
scholarships, and budgetary or funding policies. The intention might 
be changed to reflect reality, with implications for program content, 
faculty effort, and course expectations; or the admission criterion 
might be retained and enrollments deliberately curtailed, with 
implications for budget, staffing, and administrative support in view 
of the anticipated dccreas'j in student credit hours. 



As distinguished from antecedents, interactions are the actual 
encounters that represent the dynamic element of education. 
Intended interactions are the curriculum as envisioned by the pro- 
gram planners— -plans for course content, field and clinical experi- 
ences, instructional materials, readings, lectures, projects, reseaix-h 
activities, assignments, and so forth. 

Time is as critical to the intended interactions as is the nature 
of the interactions themselves, because time will influence the breadth 
and depth of the anticipated learning. The intended time allocation 
for various components of the program should match the complexity 
of the intended learning; otherwise the program will fall short of its 
goals. 

Intended interactions should have an obvious relationship to 
intended outcomes. When progi'am planners describe expected out- 
comes but fail to describe plans for interactions leading to those out- 
comes, the evaluator is likely to identify a need for information. For 
example, program planners might be requested to describe the 
intended interactions that are expected to produce an intended out- 
come. Such a description might even result in alterations of the pro- 
gram's intended outcomes through a more realistic assessment of 
curricular capabilities. 

For example, if program planners intend to train teachers who 
can effectively teach handicapped pupils in regular classrooms, then 
the intended interactions (the curriculum) should reflect adequate 
learnin^^ time, appropriate coverage of content and skills, and 
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vai'iablus ari^ piH^siint in curricular plans, neither planners nor eval- 
uators have adequate means for predicting outcomes or for suggest- 
ing progi'am modifications to alter possible outcomes. 

An additional consideration at the level of intended interactions 
is retention. Retention policies, which determine the number and 
quality of admitted students who remain in the teacher preparation 
program, are important because "the teacher preparation institu- 
tion must accept a very large portion of the responsibility for prod- 
uct characteristics upon releasing teachers as certified and 
employable" (Arnold 1977, p. 32). Measurements selected for reten- 
tion decisions (grades, grade-point averages, observations, test 
results, interviews, field monitoring, module completions, etc.) should 
match admission policies and program goals. For example, if basic 
skills are never assessed before or during a progi^am but are assessed 
in an exit examination for graduation or certification, then the pro- 
gi^am's policies do not match it:^ goals, 

The evaluator's task at this point is to identify interactions 
intended by the progi'am plarnei's, to note where modifications might 
be in order, and to begin to delineate the limits that both anteced* 
ents and interactions will impose on the intended outcomes. 



Example: Intentions --^Interactions 

Sample of evaluator's questions for identifying intemled i)itemctious. 

• What experiences are planned for students in course content, skill acquisi- 
tion, and clinical or field experiences? How does each of these relate to pro- 
gi'am goals? How will they be assessed? 

• What materials and teaching strategies will be used? Do these relate to pro- 
gram goals? 

• What projects will the students undertake? Do these reflect program goals? 
How wnll they be evaluated? 

• Do time allocations for course topics and requirements match the degree of 
complexity of these topics and requirements? 

• What retention policies will assure that suitable students remain in the pro- 
gram? What implementation procedures will show that the policies exist in 
practice as well as on paper? 

• How will unsuitable students be identified and moved out of the program? How 
will this process be assessed and made evident? 

• What is the process by which students will inform faculty of their curricular 
needs and problems? What documentation will confirm the extent of this 
process? 

• By what process will faculty evaluate students and inform them of their prog- 
ress? How will this process be documented? 
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• Is thore a meoha»ism by which students may evaluate faculty? Where will 
these evaluations be on file, who will collect them, and who will have access 
to them? 

• How will evaluation results he used? How will this usage be confirmed by later 
observers and evaluators? 



utcomes are the consequences of educating. Intended out- 



V-X comes consist of those competencies —knowledge, skills, abil- 
ities, attitudes, or aspirations — that the program planners intend 
to inculcate in trainees between entry into the program and some 
point after graduation. An orientation toward "product'' evaluation 
may lead to the atssumption that assessment of outcomes should occur 
only at graduation, certification, or during the initial years of teach- 
ing. Stake's (1967, p. 339) perception was that evaluative efforts have 
usually concentrated on such "distant outcomes." However, out- 
comes relevant to program goals also occur during the program. 
B^ormative evaluation of students' acquisition of increasingly com- 
plex skills, for example, is necessary both for monitoring their prog- 
ress through the progi'am and for continuing program revision. The 
evaluator must identify these intended, earlier outcomes in order to 
determine whether they align with program goals. Expected out- 
comes at pregi*aduation stages should support program intentions 
and provide some assurance that the program will train competent 
first-year teachers. Producing teachers who perform adequately 
during their early years in the profession may be as much as can be 
required of a traditional four-year program (McDonald 1976b; Dene- 
mark and Nelli 1981; see also Denemark and Nutter 1980). How the 
program faculty have planned to achieve specific outcomes will be of 
" particular interest to the evaluator. 

There are intended outcomes other than anticipated impact of 
instruction on students. An equally important but usually neglected 
one is the expected impact on faculty. For example, a particular 
staffing pattern might be intended to increase communication across 
lines of specialization or field; supervision might be intended to 
increase faculty's opportunities to identify research topics or offer 
inservice training. Other intended outcomes might be expected costs 
in terms of time, per sonnel, and ftinds; anticipated wear on physical 
resources; or possible effects or schools — any impact that the pro- 
gram might be expected to have on its referent groups. 

At this point, the evaluator has assessed how deliberately and 
effectively the program owners have planned the program and how 
consistent and realistic their plans are. 
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Kxamplo; lnt(MUions~<)uUonu\s 

SanipU» (il'ovaluatcjr s (|iu«slions for idiMUUy iny 'nitvmivd iHifvuHws, 

• How aiv sludonls oxpci'lcd to hv diflorunl aft(^r Uu\v haw uompkaed this pro- 
gram as I'onipaivd \u thoir vnwy chanu'lorisiu'S/skillsV How will such diftw- 
iMU'os Ik> known? 

• How will this proMrani alTect faculiy characaorisiii'S/skills^proU»ssioiml devol- 
opnu»ni? How will thost* (^tToi'ts bo JiuIkihL and by whom? 

• What will this program uost in tc»rnis of siaftlng? facilitii^s? imv and priority 
allocations? n^soarcb and sn'vk'u functions of tho institution? How will these 
costs be nu.»asurt»d? 

• What ari» the i»Np»'cicd benefits of this pro}»rani in terms of public opinion? 
reactions of teachers, adniinistratijrs, ami other school service personnel. How 
will thes(» reactions be known? 

• Where will the graduates of this projuram be enipliiyed and by whom? 



s Stake (10(57, p. 341) notod, the ovakiator has two tasks to 



perform at the; {lata-colloction stai>i?. Because examination of 
all availabk* data is impossible, the first task is selection of data for 
examination as evidence. In selecting* some data and exckidinft' other 
(lata, the (^valuator has three ^uulelines: the rationale for the pro- 
jiram, the purposes of the evaluation, and past evaluative experi- 
ences» Tho (^valuator's second task is to fit the selected data into a 
h)Mical framewoi'k: first, to identify actual conditions that need mod- 
ification and can be modified, ami second, to assess whether inten- 
tions are fulfilled. 

At the data-antecedents level, the evakiator's task — and some- 
tinu»s it is a difficult task — is to det(»rmine whether the program is 
ade({uat(» in terms of students, faculty, and resources and, if not ade- 
(jiiate, e^xactly where it is inadetjuate. 

Students 

Data availabk* to the evaluator should describe students' character- 
istics as related to proj>ram intentions. Galluxzo (U)82) maintained 
thai institutional data collection shoidd bejL>in with students* entrance 
into the proj>Tam and shoukl contimie through the follow-up phase, 
Yet Howey (Howey and Garthier 1988) noted that only 28 percent of 
institutions in a preservice study indicated that they made a formal 
preassessment of students entering' a teacher education program. 
Because the public has perceived teachers and teacher education 
students as weak, legislators in more than half the states have man- 
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dated basic skills tests as a prerequisite for entry into teacher edu- 
cation (Sandet'u)' MKSl). Projj[Tam intentions may v/ell specify particular 
student traits; yet if those traits are not systematically assessed 
among" applicants to the program, then program planners have noi 
provided basic conditions for achieving the progi'am's goals. 

Data on students' characteristics will be provided by student 
records, tests, high school and college grades, success in prerequi- 
site courses, interview results, standardized test scores, and other 
available information. The evaluator determines whether the* stu- 
dents admitted to a program match the progi'am's intended student 
body and whether the stated admission and retention policies are 
operative and result in the desired student characteristics. Lack of 
congiiience is indicated either by the exclusion of students who appear 
to have desirable charactenstics or by the admission of students with 
characteristics inconsistent with the program's goals. In the proc- 
ess of tracking congruence among program elements, the evaluator 
may uncover inadequate mechanisms for implementing intended pol- 
icies. For example, an admission system that relies on individual fac- 
ulty to identify students in their classes who have not been screened 
for eligibility will not work if a number of faculty forget to do so. 

A system that makes many exceptions or provisional admis- 
sions will thwart the program planners' original intentions. Data 
relating to the processes for screening and admission, student per- 
formance, adherence to prerequisites, testing progi'ams, and any 
other control points where student characteristics are pertinent to 
program goals will be reviewed by the evaluator. A discrepancy 
between evidence and intentions could indicate a discrepancy between 
real and stated intentions, problems in the admission and retention 
system, or problems in the system for data collection. The evaluator 
reports any lack of congiiience and attempts to determine the causes. 

Example: Data — Antecedents (Students) 

Sample of ovaluator^s (juestions for identifying data on antecedents pertaining 

to f<tit(IC)ifs» 

• What (luantitative data (ACT or SAT scores, high school and college ^n^ades, 
institutional tests, etc.) describe student characteristics? 

• What qualitative data (interview reports or videotapes, letters of recommen- 
dation, biographical information, etc.) describe student characteristics? 

• Do these data describe the characteristics specified by program goals? 

• Does the program attract the students envisioned by the planners? 

• Which students have been screened out and at what points in the progi'am? 
By what means? 




• What aiv tlu» liinils to tlu» proyi'amV ability to attract ami retain tht» desircul 
studcMits? 

• If there is conflictin^j evidence, where are the discrepancies? 

Faculty 

Kvidenee on antecedents pertaining* to faculty includes faculty vitae; 
teachinji' evaluations given by students and peers; evidence of 
involvement in professional organizations and activities; and inter- 
views with faculty, administrators, students, and graduates, 
Depending on program intentions and the purpose of the evaluation, 
the evaluator may decide to investigate the communication skills of 
faculty, as evidenced in publications, presentations, syllabi, lec- 
tures, and interviews. For example, faculty ability to link research 
to teacher education progi'ams and to convey this linkage to students 
may be assessed through syllabi, interviews, and class observa- 
tions. Relevant data may include faculty workload, policies for 
recruiting and hiring, internal governance, and opportunities for 
professional development. The evaluator determines whether the 
program faculty exhibit the professional background and pedagog- 
ical skills re(juired by program goals, 

Kxampli»: Data— Antecedents ( Faculty) 

Saniplt* j)t' evaluator s cjiu»stions for identifying data on antecedents pertaining: 
to tavnltif. 

• What are the actual conditions of faculty employment and workload? How do 
theso I'umpare with policii^s stati>d in catalogs and handbooks? 

• What data (U»scribc» the characteristics of faculty in this prugi-am? 

• Do eurn»nt faculty exhibit those characteristics described in the program's 
intentions? 

• Is \hv\v ('Videnci» that faculty who lacked desired chai\icti»ristics were removed 
from thi» program or were provided with appropriate professional develop- 
ment opportunities? 

• What faculty havt» recently been added to the progi'am? Who has recently left? 
What pro^n ani ^aps have b(»en filled in, and what gaps have been created? 

• I )o faculty percei\-e a conflict bet ween proy^i'am needs and institutional g(Kils? 

• Do faculty indicate an awareness of student needs, «;oals, and progress? 

• How do faculty intend to use program evaluation data? 

Facilities 

The evaluator will examine evidence indicating whether facilities 
(space, accessibility. tMjuipment, maintenance, and staffing) provide 
the support needed by the program, Although existence and acces- 
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sibility of resourcos are Important, the evaluator will also look at 
extent of use, budgeted support, and planned resource allocations. 
Progi'ani evaluations focusing solelj' on outcomes are likely to neglect 
such antecedents, even though they are a critical factor in the ability 
of a program to implement the intended curriculum. 

For example, an essential antecedent for training prospective 
teachers to use computers in instruction would be the availability of 
the necessary hardware and software. The evaluator will determine 
whether these materials are available, accessible, and operational. 
Questioning to elicit evidence of planned support for programs is 
intensive, focused (some might say picky), and quite possibly per- 
ceived as somewhat threatening by faculty and staff: Show the posted 
laboratory hours for the computer room. What assistance is avail- 
able for students during these lab hours? Which faculty with exper- 
tise in computer usage train the students? What time outside the hours 
of instruction do faculty have for helping students? Show the budget 
allocations that support equipment, purchase, maintenance, and 
repair; electncal expenses; insurance; software purchases; and other 
costs in operating a computer lab. What security system will curtail 
theft and vandalism and permit the lab to be used at night and on 
weekends? What hours of operation are most convenient for the pro- 
gi-am's prospective students? What is the cataloging system? How 
is software usage recorded? 

Lack of congruence between program goals and existing facili- 
ties indicates either that evidence is incomplete or that the program 
intentions are not appropriately supported. The evaluator's task is 
to note the discrepancies and the need either for revision of program 
goals or for modification of the factors limiting the attainment of pro- 
gram goals. The evaluator's t\mction is not to make decisions about 
modifications or revisions (although recommendations are usually in 
order) but rather to point out the fact and the causes of incongru- 
ence. 



Example: I )Hta— Antecedents (Support Facilities and Personnel) 

Saniplo of I'valuator's (juestions for identifying data on antecedents pertaining 
to mipport fucililies ami pem)nnel, 

• Is the necessary equipment for the program available? accessible? opera- 
tional? 

• Is the required staff in place? trained? capable? 

• Are needed support and maintenance budgeted for? 

• What is the funding status of the program? 

• What support for the program is provided by the local school system? 
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• Are the availabli' fiold sites appropriatf foj- program nucMls? How is this 
appiH)priatt'nt'ss (letonnint'd. and by whom? 



DATA— INTERACTIONS 




bservations of interiictions should produce data illustrativt' of 



two distinct areas of interest to program faculty and to the 
©valuator: intended occuiTences and unintended occuiTences. Intended 
occurrences are documented through inventories, questionnaires, 
interviews, reports, course descriptions, syllabi, and actual obser- 
vations of classroom and field situations. In selecting evidence to be 
considered, the evaluator must avoid assuming that the only inter- 
actions to be observed are the formal, planned activities as outlined 
by a syllabus. As Patton (1980, p. 147) noted, often "a major part of 
a progi-am's impact may take place on the periphery of structured 
activity A number of factors may cause the curriculum not to be 
implemented as it was intended, for better or for worse. The dynam- 
ics of the classroom situation*, students' reactions to readings, activ- 
ities, and discussions; and extracurricular activities may well affect 
tne progi'am. Variations in course staffing and sequence, quality of 
instruction, instructors' contact with students outside the class- 
room, and interaction with field sites can all increase or reduce cur- 
ricular impact. Travel time to field sites may significantly reduce 
students' field contacts; a gifted instructor may increase students' 
motivation. Part-time instructors may not understand how a partic- 
ular course relates to the total progi-am and so may downplay some 
critical cuiTiculum components. 

A valuable source of data on unintended occurrences is class- 
room observation. Documentation of such observations might include 
descriptions of the dynamics of classroom experiences, the duration 
of specific activities, the frequency of particular teaching strate- 
gies, the sequence of content and skills practiced, the nature of stu- 
dents' participation, and analysis of the levels of skill and knowledge 
acquired by different students. 

Unintended occurrences may also be identified through student 
and faculty feedback. Data that are typically missing from program 
evaluations are student and faculty perceptions of what is really 
happening and how valuable these perceived happenings are. While 
faculty have input into a program and can plan changes based on 
personal observations, student evaluations, with some notable 
exceptions (see Sorenson et al. 1977), seem to be used solely for 
assessment of instructors. However, students' perceptions of their 
program have direct bearing on what role they assume in the pro- 



36 



ERIC 



44 



H'rani and, evontually, their status as jyi'aduates of the proj^i'am. For 
example, students who are unaware of the reasons for particular 
assiii'nments or experiences are likely not to value them, Modifica- 
tions, in this case, would consist of clearer communication of pro- 
tlframs and strategies. 

Reliable student feedback, positive and negative, is critical to 
pi'ogram evaluation. I\*rsistent negative responses from students 
about any program component indicate a need to reassess that com- 
ponent. Persistent positive responses may suggest strategies or 
activities that would enhance other progi'am areas. 

Data from progi'am graduates and their supervisors, princi- 
pals, and colleagues are also extremely valuable. Because these clients 
base their perceptions on the actual performance of program grad- 
uates, their responses provide critical data for progi'am review and 
revision. The evaluation should ensure that program owners know 
what significant groups of clients think about their program, and the 
program should be amenable to modifications based on that feed- 
back. 

Data on graduates' performance must be interpreted in the 
context of local factors beyond the control of the preparation pro- 
gi*am. Teachers are vulnerable t pressures to display site-specific 
behavior patterns not necessarily in conformity with the content of 
their training (McLaughlin and Marsh 1978; Squires, Huitt, and 
Segars 198!^; Smith 1980). Fbr example, a program that trains future 
teachers to use a language-experience approach in reading and writ- 
ing might conflict with school systems that restrict teachers to using 
manuals and standard textbooks. On-site assessments of these 
teachers wouki reveal little of their language-experience training, and 
an inexperienced or uninformed observer might judge the progi'am 
to be ineffective. In a case such as this, the evaluator will point out 
the incongi'uence between the program's goals and the school dis- 
trict's expectations. The evaluator might also recommend to the 
program faculty that if the language-experience training is to con- 
tinue to be a primary focus of the program, then the skills of grad- 
uates would be more appropriately evaluated in other contexts. 

Scriven (1972) recognized that stated goals may mask unin- 
tended interactions and advocated that evaluators not consider offi- 
cial goals as a starting point. Rather, he suggested, evaluators should 
focus on actual occurrences and effects and use these to formulate 
reality-based program goals that can then be compared to offirial 
goals. This comparison of evaluation-deduced goals with stated pro- 
gram goals would immediately highlight areas of incongruence. 
Scriven's strategy has the disadvantage of leaving the evaluator with 
no guidelines for selecting the data to be used in making judgments. 
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Howovor, i»\pf!Men(fi»<l i^valuators are awaru of thu tunnel vision that 
results troni focusing* only on official goals. 

The obsei'ved interactions in a teacher p!'i?paration cui'riculum 
are likely to be less extensive than the specified intended outcomes 
would suggest This common discrepancy derives tVom the fact that 
teacher e(iucators typically set goals that, while accurately reflect- 
ing the knowledjie ami skills beginning teachers need, far exceed what 
is possible to do well in the time permitted under existing course and 
credit constraints (I)enemark and Nutter 1980), Therefore, the evaK 
uator must expect some discrepancy between intended interactions 
and observed intei'actions. It is unlikely that all program intentk)ns 
can or will be realised. The evaluator will focus on the extent, direc- 
tion, and impact of expected discrepancies* rather than on the fact 
that discrepancies exist. 

Another negU?cted area is that of attrition data (Nutter 1983). 
The evaluator should ask how many students leave the program and 
why they leave. There are always social, personal* and economic trea- 
sons for leaving school. There may also be reasons relating to pro- 
gram content, course sequence, or faculty expectations or 
charactenstics. The evaluator's task is to bring these factors to light. 
If such data do not exist, the evaluator must determine whether the 
lack derives from high retention rates or failure to collect informa- 
tion. 

Where data on interactions conflict with intentions, tiie evalu- 
ator must note the discrepancy. A teacher preparation program that 
intends that all students will master the basic skills of classroom 
management presumably will provide for the development of these 
skills within the planned curriculum. However, if the data on inter- 
actions show that students receive little instructi(m other than com- 
monsense admonitions, no classroom observations focused on 
management skills, and no opportunity to practice these skills before 
student teaching, then intentions and interactions are incongruent. 
The evaluator would note the discrepancy and suggest possible causes. 

As Stake (1967) noted, people differ in what they value in a 
training program. Some will discount the importance of congi'uence 
between intentions and interactU)ns; others will consider it to be crit- 
ical- Students, administrators, faculty, and the public may well have 
different perceptions of the importance of a specific discrepancy The 
evaluator's statements, while reflecting some degree of personal 
judgment, should be g)'ounded in an understanding of program goals 
and the purpose of the evaluation, for these provide the evaluation 
parameters. 
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Kxampio: Data— InU'raotions 

Sample ol" I'Valuator's (lui'stions Cor idoiuit'yinjs data on iiiUradions. 

• What aiv tlu> principal inti'ndod liUoractionj*? How is thoir occurronco docu- 
monted. and by whoni? 

• What unintended interactions ocoiir persistently? How is tlieir occurrenct! 
t'videnced".' 

• What do tiU'ulty say about the benefits or drawbacks ot'tho coursework in this, 
pn)jsrani? Mow do they make their perceptions known? 

• What do current students say about their experiences in the pr()ni'am? When 
do they communicate their reactions, and to whom? What effect do these 
reactions have on course content? 

• What do pro|L!,'rani jrraduates and other school-based professionals say about 
what is tauu'ht in this projiram and/or how it is taujiht? When d" they com- 
municate their reactions and to whom? How are these reactions rotlected in 
pro>>ram interactions and outcomes? 

• Are there discrepancies between the content of the pronrani and expecta- 
tions for its ji-raduates? What are they, who documents them, and what has 
been done to resolve them? 



he definition ol'ontvnnte is critical. To many, the only "out- 



come" is at the point of p-aduation and during initial teaching'. 
Most program evaluation has concentrated on this view of the out- 
come level. If, however, as Arnold et al. (1977) suggested, selected, 
se(iuential, quality-control points are seen both as outcomes of pre- 
ceding quality-control points and as entry requirements for suc- 
ceeding point's, then there is a variety of outcomes for which data 
should be sought (Medley 1982; Roth 1982). Eight "key (luality con- 
trol points" are evident in teacher preparation: 

1. College admission 

2. Admission to the teacher education progi'am 

'A. Student teaching and other professional field experiences 

4. Completion of preservice preparation (graduation) and insti- 
tutional recommendation for certification 

5, State agency certification 

(). Employment (initial year or years) 

7. Retention and tenure decisions and inservice education 

8. Continuing professional development (Arnold et al. 1977, p. 7) 

Whether considered to be integral portions of programs or sep- 
arate levels, the first four points are unquestionably the responsi- 
bility of teacher preparatlj^n programs. Of these, student teaching 



DATA— OUTCOMES 




47 



ERIC 



is the (lultnination of the projn'am and traditionally has been judftvd 
by students as the most potent part of the proffj'ani (Haberman 198;^). 
Student teaching has been the subject of numerous investigations 
(e,gM Canadian Teacher's Federation 1977; Cortis and Grayson 1977; 
Dickson and Wiersma 1980), Evaluative data from student teaching 
have two benefits for the training institution. First, students are 
assessed at their optimal level of pregi^aduation knowledge and per- 
formance skills, and so results are more directly attributable to the 
program, Second, student teachers are accessible; data can be col- 
lected far more easily before gi'aduation than afterwards. 

Data from teacher-effectiveness research (e.g,, McDonakl 197()a, 
197(>b; McDonald and Elias 1976), school-effectiveness studies (e.g. , 
McLaughlin and Marsh 1978; Squires, Huitt, and Segars 1988), and 
pupil-achievement research (e,g., Berliner 1976; Evertson, Ander- 
son, and Brophy 1978) can and should influence teacher education 
progi'ams. There are, however, reasonable limits to the impact of such 
results for an individual program on evaluation, 

Patterns of strengths and weaknesses demonstrated over time 
by graduates from a particular program may indeed point to pro- 
gi'am strengths and weaknesses (in admission, training, gradua- 
tion, or evaluation practices) for which the program must be held 
accountable. However, the tjvaluator must interpret such data judi- 
ciously in light of the contexts in which graduates work. Teacher 
preparation has no control over policies and procedures operating in 
most school systems. Peer pressures, administrative expectations, 
recogniti(m, rewards and punishments, parental influences, and 
community input all have an impact on teachers, and, singly or in 
combination, alter teacher behaviors (McLaughlin and Marsh 1978). 
A preparing institution should not be held solely accountable for 
behaviors that are the result of postgraduate pressures and expec- 
tations. 

Systems foi* follow-up and evaluation of graduates are ciiicial tor 
program improvement. There are many follow-up models available 
(e.g., Sorenson et al. 1977; Oana et al. 1975; Adams 1978). It seems 
clear that multiple measures of giwluates' performance are more 
effective and useful than are single measures (see Sandeftir 1982; 
Savage 1981; Yeany 1980). At the outcome phase, the evaluator's 
selection of data is again critical; and, again, selection will be based 
on the evaluation's purpose and the program's rationale. The evalu- 
ator may supplement questionnaire data with on-site observatitms 
and interviews with selected graduates, colleagues, and employers 
of graduates to provide a stronger base for evaluation. 




Kxample; l)utu~()utcomes 

Sample? of ovaluatorV (luustions for iilontifyinK data on imfvimivs. 

• What (lata aiv availablo on pn)HTani K^'t^duatos? Who C'olU?cts thesu (hita? Who 
m'olves tho Information? Dous this information aftbct pro^'am content? How 
art? offocts (U)cumento(l? 

• What stiulonts loave this projvram? Why do thc\v leave? What documentation 
is thero of studont-attrition rates and causes? 

• What knowledge and sl<ills does this pi'o^ram develop in students by the point 
of graduation? How are these evidenced? How are results fed back into the 
program? 

• How do principals, teachers, and other professionals evaluate the total pro- 
jrrani? How are these data collected? What evidence is there that they have 
an impact on the program? 

• Are other data available concerning uraduates of the projji'am (standardised 
tests, competency tests, etc. )? How do the results of these tests support pro- 
t^ram intentions? 



t the intentions and data levels, the evaluator focused on the 



JLJLcharactenstics of the students, the faculty, the cumculum, and 
the supporting facilities. At the standards level, the focus is on whether 
the real status of the program meets conditions required by the 
standards. 

In relating written standards to on-site evaluations, the evalu- 
ative process becomes a determination of priorities (see Figure 2) in 
which there are four consecutive levels: selection of standards, appli- 
cation of standards, judgments about the program, and dissemina- 
tion of the judgments. The first two levels, selection and application, 
are part of the standards process. The last two will be discussed under 

Selection 

Everyone has standards to apply to teacher preparation programs, 
An aroused public, or concerned gi^oups within the public, may demand 
that absolute and very specific standards be mandated for teachers 
and for teacher preparation. More directly and continuously related 
to teacher training are the national, state, and regional agencies and 
professional organizations that have assumed responsibility for 
monitoring those programs and enforcing or urging adherence to 
guidelines. Additional strictures are presented by federal, state, or 
other regulations that set conditions for funding. Finally, adminis- 
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trative regulations within colloge.s and univorwsities set ^mxh and limits 
for all institutional programs. 

For any |?iven progi'am, the referent gi'oups directly involved mi\y 
devise standards relative to the specific setting and area of training. 
These relative standards mwy be more familiar to faculty and stu- 
dents in a program than are the absolute standards on which ac- 
creditation, funding, or public opinion are based. 

In practice, the distinction between relative and absolute 
standards is likely to blun Many program planners routinely con- 
sider absolute standards (such as state regulations and the progi'am 
guidelines of professional organizations) when they modify and update 
programs for in-house purposes. A blend of absolute and relative 
standards is exemplified in the field-based Chicago State University 
evaluation model described by Dillon and Starkman (1981). This model 
was created both to meet NCATE and state accreditation require- 
ments and to provide in-house data on program effectiveness. 

Stake's (19()7, p, 344) model for assessment of a program distin- 
guished between relative and absolute comparisons. For program 
improvement based on formative evaluation. Stake's model used 
descriptive data from other progi'ams to provide relative compari- 
sons. Formal standards from professional gi'oups and agencies were 
used for absolute comparisons of progi^ams in a summative evalua- 
tion. 



Application 

Applying the selected standards is a comprehensive process, first, 
of examining the antecedents, interactions, and outcomes as evi- 
denced by the selected data; second, of fitting these vi\riables into a 
logical framework; and third, of putting this ft'amework in the con- 
text of the standards. Implementation of the standards requires 
consideration of evidence in multiple areas: curricula, students, fac- 
ulty, funding sources, planning effectiveness, facilities, community 
support, cost effectiveness, program goals, field placements, out- 
comes, organizational efficiency, progi*am autonomy, and any other 
components deemed important by the initiators of the evaluation. Some 
of these variables will indicate quality levels for which a progi*am is 
held accountable* and some will represent goals toward which the 
program is oriented. 

At the point of application, most data relating to antecedents, 
interactions, and outcomes will be in place, ready for the process of 
review. The actual standards to be used have been determined, and 
questions relating to their implementation deal with the extent to 
which specific standards should be applied. It may be that some 
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Figure 2: Levels of the Standards and Judgments Stages 



Referent Groups/People Modifying C:onsiclei'ation8 



SELECTION 



APPLICATION 



Pro^yrafn faculty 
Accreditation agencies 
Professional associations 
Graduates of program 
Teachers 

School administrators 
University administrators 
The public 
State government 
Federal government 
Others 



Evaluator 



POSSIBLE STANDARDS 



Purpose of evaluation 
Natuiv of the program 



STANDARDS 
FOR THIS EVALUATION 



Competing or conflicting standards 
Program contingvncies 
Pi'ogram congruence 

"Real" vs. **|)apei''* constraints and support 
I^i'ogram goals 



JUDGMENT 



PROGRAM IS ADEQUATE OR 
DEFICIENT IN EACH OK 
THE STANDARDS 



FEEDBACK (USE) 
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Dissemination of judgment to program faculty 
and to other appropriate reference gi'oups 
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standards should hv emphasized ovei* others, depending on the pur- 
pose of the evaluation and the results of previous evaluations. Where 
deficiencies have been noted in the past, current evaluation ojfforts 
may well focus on the extent of improvement in key areas. 

One immediate consideration will be the nature of the data that 
have been collected. If data ai'e restricted or contradictory then the 
evaluator may wish to inquire further about reliability and validity 
Meeting a standard must be more than juggling data in order to claim 
paper compliance. 

The evaluator will also want to know how aware program own- 
ers are of particular standards and what steps they have taken to meet 
them. F^aculty and administrators can often use accepted standards 
as leverage to obtain program support; have they done so? For exam- 
ple, administrators aware of state regulations for library facilities 
coukl, in the months preceding an evaluation, use administrative 
channels to predict adverse evaluation of existing support for the 
library. A central administration anxious to avoid the appearance of 
short-changing future teachers may well allocate additional funds. 
Similarly faculty slow to review curricula or develop follow-up pro- 
cedures may take action under the pressures of having to meet 
standards for an imminent evaluation. Attention to standards and 
thoughtful responsiveness is one indication of administrative plan- 
ning effectiveness. 

Characteristics of students, faculty; and facilities and support 
personnel have been noted as critical in any evaluation of anteced- 
ents, interactions, and outcomes. All evaluation standards dealing with 
program effectiveness will refer to students; most will recognize the 
importance of faculty, and many will refer to support for programs. 



Student characteristics 

The most obvious antecedent, admission requirements, could be 
program-specific, institution-mandated, required by the state, 
written into professional guidelines, or a combination of these. The 
evaluator focuses on whether the program uses appropriate infor- 
mation to admit students who are most likely to benefit from the pro- 
gram and become competent beginning teachers. 

Standards for student interactions relate to required course- 
work, sequencing of experiences, duration of activities, feedback on 
performance, and student perceptions of the importance and effec- 
tiveness of what they are doing. The evaluator will be interested in 
the categories of costs that have an impact on the quality of interac- 
tions (Thompson et al. 1981). How local school districts assess the 
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pvoftTam, and hy what standards, will also tall within the pur\ iuw 
of the evaluator. 

Standards for student outcomes exist in the tbrm of graduation 
criteria for knowledge' and performance skills; criteria for evaluation 
by peers, instructors, school administrators, and the public; district 
hiring' policies; employment statistics; and state regulations for ini- 
tial and emergency certification. 

Kxaniple: Standards (Students) 

Sample (jf evaluator s (jiu^stions ihv darityinji* the fit between standards and 
program realities eoneerninji* sfiulvnts. 

• Does the pro^Tam a|)|)ear to he e()lleetinji>' and usin^' appropi'iate data to iMUorce 
admission and intention criteria? Are |)r()B*ram criteria and efforts consistent 
with standards relating to student characteristics? 

• Are the data sufficient to indicate whither the prufjram meets curriculum 
standards for students? 

• What data show the tjUaiity of y'raduatinji* students? Do thesi» data sh(Av 
whether exit standards have been mvV! 

• Were wvaknesses or deficienci(»s relating to tlie student body identified |)rior 
to this evaluation? If so, have these bet»n adchvssed in such a way as to meet 
the appropriate standards? 

Faculty characteristics 

Standards relating? to lactdty (lualificatitms and abilities vary in 
emphasis on the teaching, service, and research efforts ot'instrtic- 
tors. A school district might have a standard for faculty's service^ 
involvement that conflicts with an instittitional standard for publi- 
cation. For a professional organization, program quality might be 
evidenced by presentations of reseaivh papei's at conferences; but tor 
prospective students, by awards for superior teaching, 

Documentation of faculty quality can be providi^d through stu- 
dent evaluations, peer reports, observations of teaching, service and 
inservice records, affiliations and activities with professional groups, 
committee activities, publication, i)apers presented, funded research, 
number of citations by other scholars, and other evidence important 
to the initiators of the evaluation. The evaluator must i>\amiiu> these 
data for reliability, currency and ade(|uacy as evidence of meeting 
the selected standards. 

Example: Standards (Faculty ) 

Sample of evaluatorV (juestions for clarifying the fit between standards and 
profrram realities conccrninf^./rKv/////, 

• Is the pool of data on faculty sufficiently curivnt . reliabU^ and comprehensive 
to permit standards to be applied? 

15 
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• What (lata have boon collected as evidence of continuing faculty ability and 
effectiveness? Are these data sufficient for showing whether standards have 
been met? 

• What evidence shows faculty involvement in planning, implementing, and 
evaluating programs? Is the evidence adequate for showing whether stand- 
ards are met? 

• Were weaknesses or deficiencies relating to faculty identified prior to this 
evaluation? If so, have tiiese been addressed in such a way as to meet the 
appropriate standards? 



Facilities and support personnel 

Standards relating to funding formulas are of critical importance. They 
provide not only goals for effective operation but also impetus for 
improvement. The same is true of standards relating to the amount, 
allocation, an(l number of support personnel. Although a progi'am 
faculty and administration can determine critical funding parame- 
ters by rc?viewing relevant literature and consulting with experts 
(Yeaton and Reciner 1981), the possibility of having a progi'am judged 
deficient may have greater impact on reluctant budget offices. If 
funding is a primary focus for the evaluation, the evaluator might 
recjuest additional information in terms of direct and indirect insti- 
tutional stamiards (Thompson et ah 1981). The evaluator is respon- 
sible both for noting incomplete, outdated, or biased data that prevtmt 
the application of standards anci for requesting additional data as 
needed for judgment. 



Kxample: Standards ( Facilities and Support Personnel) 

Sample' of wiluator's (juej^tidus for clarifying tho fit botwctMi standards and 
pro)>*ram realities concorning/ff('///7/V.s ami .s///i/io/'f iwrsuhnvL 

• Aw the available data on facilities, services, and support personnel suffi- 
ciently current and reliable to permit application of standards? 

• Are the data on funding formulas (per student, per ci'edit hour, per type of 
instruction, per level of student) sufficient to permit application of appropri- 
ate standards? 

• What data have been collected on support for progi^am needs in materials, 
facilities, services, and support personnel? Are these ade({Uate for showiuK 
whether standards have been met? 

• Were weaknesses or deficiencies relating to facilities, services, or personnel 
identified prior to this evaluation? If so, have these been addressed in such a 
way as to meet the appropriate standards? 
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JUDGMENTS 



In a sense, many se^yments of the public have ah'eady judged teacher 
preparation and the teaching profession and found them defi- 
cient. It is common to read or hear remarks such as those made by 
William J, Bennett, chairman of the National En(iowment for the 
Humanities, who, in a speech to the Amencan Legislative Exchange 
Council (^'Humanities Chief . . 1983, p, 1), charged that "people 
who have an education degi'ee may teach history, but people who have 
a history degree or deep knowledge of history may not teach it.'* 
Apparently, ignorance about teacher preparation is no deterrent to 
criticizing it. 

Critics of teacher education are often unaware of the antecedent 
conditions that limit and direct teacher preparation programs and of 
the variety of data that can be used as measures of program effec- 
tiveness. Often, they resist learning anything that runs counter to 
their prejudices. Legislative committees that have not sought input 
from teacher educators recommend legislation designed to have major 
and lasting effects on teacher preparation progi'ams. The same com- 
mittees often neglect to recommemi the funding required to imple- 
ment changes, ami the resulting legislation may actually weaken 
program effectiveness. 

Teacher educators need to create a new standard for them- 
selves, one that woukl hold them accountable for initiating and main- 
taining information systems that reach extensively into public arid 
professional sectors. 

At the same time, it must be noted that judgments of program 
inadequacy based on antece(ient comiitions beyond the control of 
teacher preparation units are the most palatable to progi^am faculty 
and administrators, A deficiency traced to a source outside the 
preparation program puts less pressure on faculty to act and 
encourages the luxury of complaint without the burden of initiative. 
Far less palatable are judgments of inadequacies that stem from what 
program owners have or have not (ione, Because of the cuirent stress 
on accountability, teacher preparation programs are especially vul- 
nerable to judgments about systems for collecting and organizing data 
to document program effectiveness. 



Judgments about the program 

To arrive at this stage, the evaluator has determined that the infor- 
mation about the progi*am is sufficient and has compared this infor- 
mation to the appropriate standards. Judgments consist of the 
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ovaluat())''s |H>ri'i>|)ti(>n of tho extent to which antecedents, interac- 
tions, and ()Utt'(nn(.'s match the relevant standards. 

JudMinents can be provided in brief statements such as "ade- 
(luate" or "inadeciuate." or elaborated to specify areas of deficiency, 
adecjuacy, and strength. In times of economic uncertainty, judg- 
ments relatinM" to program effi'ctiveness may have a particularly 
poNverful impact, since they provide documentation tor decisions about 
pj'iorities and funding. Such decisions can result in maintaining or 
eliminating a program. 

The ('Valuator may be charged with supplying recommenda- 
tions for program improvement or may (AevX to do so. Such recom- 
mendations must outline steps by which program weaknesses might 
be remedied, alternatives for program improvement, or suggestions 
for maintaining exemplary levels of a(;hievement. 

KxampU>: .lud^mentH 

Saiiipli' ori'valuatorV qiit'-^tioiis to an-iw at 

• How flosoly (Id the pi-oKvamV aiitcu'cdents, iiUt.'ractioii.««. and outcomes match 
the appi'opi'iati' standards? 

• Are tlu« program data fcmsistcnt with thc> standards for curriculum interac- 

tidMS? 

• Do intondcd unintended oulcomcs .streniilhen or weaken the effectiveness of 
the pi'ojrram? 

• Whifh standai'ds ai e inadoiiuatel.v met in the progi-am and why? What appear 
to \w sourc-cs of deficioiU'les? 

• Which standai'ds ai-e met acU-qiiately, and how could they be met more strongly? 

• Which standards are met in an exemplai-y wa.v. and what are the sources of 
these sli-enKth.s? 

Disstimi nation of jud^fments 

The evaluation process is a useless exercise unless the results are 
(••ommunicated to appi-opriate groups. It is difficult to visualize an 
evaluation process that omits program faculty from among those 
ivceiving the results because program modifications based on eval- 
uation .ire an ongoing professional obligation. There will often be other 
groups w ith an interest in or need for the evaluator's judgments about 
the program: administrators with budgeting priorities to set; school 
districts w ith hiring decisions to make; accrediting agencies; and 
others. No group. hov\ever, has more intense interest in, or gi-eater 
need for. the results than do the faculty and administrators respon- 
sible for the program. 

The w ay results are (li.^.^eminated will depend on requirements 
s))ecified by the initiators of the evaiuation process. Some evalua- 
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tions require extensive reporting; others require less. It is the eval- 
uator's responsibility to provide judgments in whatever format is 
required. Ideally, evaluation results v^'ill be reported in such a way 
as to be useful for all interested groups, The process itself, if carried 
out with integrity and intelligence, leads not only to an improved 
progi'am but also to increased understanding of the achievements of 
teacher educators. 

THE MODEL APPLIED 

The following examples demonstrate how the model might be 
applied in actual evaluations. 
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GOAL: Provide a high-quality school-media progi'am 

ANTECEDENTS 



Example 1 

that will train students in media/technology for use in schools. 



Intentions There will be sufficieiit media facil- 
ities for quality media/technology 
training. 



Data P'acilities are available. Schedules 

are appropriate. Equipment is old; 
and, because of expense, mainte- 
nance contracts have been allowed 
to expire. (Inspection of facilities, 
inventories, budgets; interviews 
with staff and faculty) 

Standards Appropriate a-v equipment will be 
accessible to students. Trained fac- 
ulty/staff will direct students. Lab 
hours will be appropriate for stu- 
dent use. 



INTERACTIONS 

Quality clinical experience will be 
provided in courses and lab hours. 



Students sign up for a-v courses, 
attend classes, and sign up for lab 
hours. However, equipment contin- 
ually is inoperable because of main- 
tenance problems. (Observations, 
interviews with students) 

Students will be provided ample 
opportunity to use a-v equipment 
under well-trained instructors. 



OUTCOMES 

Graduates of the program will be 
competent to plan instruction using 
media and to operate all audio-vis- 
ual e(iuipment found in typical pub- 
lic schools. 

Students are deficient in the amount 
of time they are able to spend with 
a-v equipment. Students are not 
ftUly competent in operating e(|uip- 
ment. (Interviews with students, 
questionnaire "esponses from 
graduates and their supervisors) 

Graduates of program can plan for 
the use of a-v equipment in class- 
room and implement and evaluate 
their plans. 
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Judgments Program is adequate. 

Antecedent conditions are likely to 
hamper ability of program to pro- 
vide enough hands-on opportunities 
for students. 

Equipment availability must be 
monitored. 



Program is deficient. 

Data show that students do not have 
sufficient hands-on practice. The 
program lacks congruence between 
intended and actual interactions with 
a-v equipment for students. 



Program is seriously deficient. 

Graduates are not fUlIy competent 
in operation of equipment^ although 
planning for its use and evaluation 
of use in schools is adequately cov- 
ered in coursework. Students in 
schools arc able to draw on experi- 
enced personnel there to assist them 
with running the equipment. Stu- 
dents evaluate equipment usage. 



Evaluator^s Recommendations: 1. That information about the program's deficiencies in this area be communicated to administra- 
tors with budgetary responsibility 2. That the program faculty actively seek an appropriate realignment of budget allocations to 
support the media component properly 



Example 2 

GOAL: Graduates will be competent in written communication; they will write fluent, coherent, well-organized prose with stan* 
dard English usage and acceptable spelling and punctuation. 



Intentions 



Data 



ANTECEDENTS 

Two courses in English composi- 
tion completed with giwles of (' - or 
l)etter piior to entry intothe teacher 
education progi^am will ensure that 
prospective teachers have adequate 
command of written communica- 
tion. 

The selective admission system 
depends on instructors in the edu- 
cation department to identify stu- 
dents who have not been checked for 
eligibility. A number of instructors 
forget to do so« and ineligible stu- 
dents regularly complete the pro- 
gram. (Inspection of records, 

interviews with faculty and staff) 
* 

G!*ades in the composition courses 
run high, and very few students earn 
gi*ades lower than C' - . (Inspection 
of records winter views with faculty 
and staff) 



INTERACTIONS 

Instructors in the English Depart- 
ment will teach the full range of 
v\'nting skills to a level acceptable for 
an.v college graduate, 



The English department's philoso- 
phy stresses personal expression 
and creativity in writing. The 
graduate assistants who teach the 
freshman composition courses are 
told not to teach or evaluate usage, 
spelling, or punctuation, but to refer 
deficient students to the develop- 
mental center for tutorial assist- 
ance, Students are urged, but not 
recjuired, to \4sit the center (Inter- 
views with English department 
staff) 



OUTCOMES 

All graduates will be c(nnpot(?nt in 
the writing tasks commonly 
expected of teachers and will have 
the level of writing skill appropriate 
for professionals who interact witli 
children, parents, and the general 
public. 

Instructors in the education 
department complain strongly that 
many of their students do not write 
well enough, They express consid- 
erable frustration over the situation 
and fe^l they do not have either time 
or expeitise to teach basic writing 
skills, 

Cooperating teachers express dis- 
satisfaction with the writing skills 
of some student teachers and report 
that these student teachers cannot 
acceptably complete basic writing 
tasks— lesson plans, written mate- 
rials for pupils, reports, communi- 
cations to parents, etc. (Interviews 
with faculty, coopei^ating teachers) 



Standards Tho criteria and criteria levels for 
admission to the teacher education 
prop^am will select only those stu- 
dents with the desired characteris- 
tics. 

I'he selective admission system will 
operate to apply the specified cri- 
teria to all candidates for admission 
to the program. 

Judgments Program is deficient. 

It lacks congruence between the 
intended criteria and what the cri- 
teria actually reflect and between 
the intended and actual system for 
assessment. 



The instructional component linked 
to the selective admission system 
will in fact address all the compe- 
tencies the program owners believe 
shoukl be addressed. 



Progi*am is deficient. 

The English department's percep- 
tion of its mission is a contingency 
that powerfully limits the pro- 
g)*am's attainment of this goal. 



Students in the program will diwS- 
play competence in writing in their 
course assignments and in their field 
assignments. 



Program is deficient. 

It lacks congruence between the 
intended outcomes and actual out- 
comes reported by instructors and 
school personnel for a significant 
number of students. 



Evaluator's Recommendations: 1. That the program faculty work, perhaps with the English department, to develop an alternate 
system for assessing basic writing skills and for requiring retnediation for students who are deficient. 2. That the administrative 
procedures for selective admission be redesigned to ensure that all students are checked for eligibility at the appropriate point. 
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Example 3 

GOAL: F'kculty workload will permit adequate time for instructional preparation, advising 
side of class, professional development, and research and service 



Intentions 



Data 



ANTECEDENTS 

Program faculty will be assigned 
eight to twelve quarter hours of 
classes by departmental policy, with 
attention to factors such as number 
of students, level of students, 
administrative or service assign- 
ments, and obligations for research 
and with attention to balancing 
workloads across quarters. Over- 
load teachitig will be voluntary and 
compensated. 

Departmental records show that the 
load assignment policy is reason- 
able, given the number of faculty and 
students and the non-tenching obli- 
gations of faculty, and that excep- 
tions to it are supported by 
justifications. 

Because neither institution nor 
department faculty have adopted a 
policy limiting overload, the depart- 
ment chair does not believe she can 



INTERACTIONS 

The department chair, in consulta- 
tion with program faculty, will 
assign faculty loads on an equitable 
and flexible basis and develop 
resources for part-time and adjunct 
faculty to meet special needs for 
staff. 



students, meeting with students out- 

OUTCOMES 

Progi'am faculty will have adeciuate 
time for the array of ta^sks that 
directly and indirectly promote 
quality instruction. 



The department chair follows the 
accepted policy for regular load 
assignment. She determines 
assignments in conjunction with 
program faculty 

Because the institution has no clear 
policy limiting overload, a few fac- 
ulty voluntarily accept overloads that 
appear unreasonable. These fac- 
ulty do not agree that their loads 'dxv 
excessive. 



With a few exceptions faculty have 
adequate time for their responsibil- 
ities. 

The few cases of excessive over- 
loads occur primarily in the sum- 
mer. Students report shortened class 
time, few outside assignments, and 
heavy reliance on **packaged" 
instruction and **guost" instruc- 
tors. 
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limit faculty':^ voluntary participa- 
tion in on-campus, extension, and 
inservice teaching on an overload 
basis. 

In one area, a faculty member feels 
obligated to accept unwanted over- 
load assignments to meet depart- 
mental staffing needs. (De- 
partmental records^ interviews with 
faculty) 

Standards The plan for determining faculty 
workload will permit adequate time 
for the array of tasks that directly 
and indirectly promote quality 
instruction, and extra effort will be 
voluntary and rewarded* 



Judgments Program is adequate with deficien- 
cies noted in isolated instances. 

The plan for allocation of instruc- 
tional load is appropriate. Question- 
able deviations are few and isolated. 



Except for one specialty, the chair 
and program faculty have identified 
a stable group of part-time and 
adjunct faculty with appropriate 
credentials. Designated faculty 
orient new part-time and adjunct 
faculty. (Department records; 
interviews with the head of teacher 
education, department chair, part- 
time faculty, and program faculty) 

Administrative procedures will 
properly allocate faculty time and 
will be applied consistently and 
judiciously. 

P^-time and ac^unct fbculty will be 
professionally competent, and their 
use will be monitored to maintain 
progi*am quality 

Program is adequate with a defi- 
ciency noted in one area. 

Generally, administrative proce- 
dures result in consistent and equi- 
table allocation of faculty effort. 



Instruction provided by part-time 
and adjunct faculty conforms to 
standards and curricula estab- 
lished by the program faculty. 
(Interviews with students, faculty, 
department chair, and part-time 
faculty; student evaluation of courses 
and instructors) 



F^iculty will have and make produc- 
tive use of time to prepare and pro- 
vide instruction of high quality 

Exigencies of staffing will not 
diminish the continuity and quality 
of instruction intended by progi*am 
faculty 

• 

Progi*am is adequate, but in need of 
review by program faculty. 

Overall, faculty workoad is reason- 
able, with the exceptions noted. 



Evaluator's Recommendations: 1. That the program faculty review the effects of excessive overload teaching on the integrity of 
the curriculum, with particular attention to the summer semester. 2. That efforts be made to develop additional faculty resources 
in the one specialty area through future hiring, a search for qualified personnel in local school districts, and/or professional retraining. 
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Example 4 

GOALi Students will participate in extensive field experiences prior to student teaching. Field experiences will be planned, 
supervised, and evaluated. During field experiences, students will experience specific examples of practice related to the- 
ory and will themselves practice and develop specific skills. 



Intentions 



Data 



ANTECEDENTS 

The Field Experience Office staff 
will place students in appropriate 
field sites to meet objectives deter* 
mined by the faculty 



Field assignments are made accu- 
rately and promptly Teachers are 
informed of the various field expe- 
riences through personal contact and 
printed materials; and students are 
oriented to objectives and expecta- 
tions, schools, appropriate behav- 
ior, etCi 

Systems for recording field place- 
ments, disbursing honoraria, and 
evaluating placements are compre- 
hensive and effective. 

School pen onnel report excellent 
rapport and frequent communica- 



INTERACTIONS 

Students will engage in a gradu- 
ated series of field experiences tied 
to specific course content and objec- 
tives. They will observe selected 
examples of theoretical concepts and 
practice selected skills. 



Students do in fact complete the. 
intended field experiences. 

Faculty exhibit a strong commit- 
ment to the field component and 
systematically supervise and eval- 
uate their students* performance. 

Program faculty identify students 
who experience difficulty in field 
work for remediation, counseling, or 
dismissal, as each ease necessi- 
tates. 

Students value their field experi- 
ences and report increasing confi- 



OUTCOMES 

Students will enter student teach* 
ing prepared to assume instruc- 
tional responsibility more quickly. 
Student teaching will focus more on 
refining students' practical skills, so 
graduates will exhibit a generally 
higher level of competence as begin- 
ning teachers. 

Cooperating teachers who super- 
vise student teachers report a defi- 
nite increase in the skill and 
motivation of student teachers since 
the implementation of an expanded 
field experience component. 

Recent graduates of the program 
report increased confidence in their 
ability as beginning teachers com- 
pared to earlier graduates and rate 
their field experiences very posi- 
tively. (Interviews with cooperat- 
ing teachers, follow-up surveys of 
graduates) 



tlons with Field Experience Office 
staff and effective procedures for 
dealing with problems related to field 
experiences, 

When appropriate field sites are not 
available locally, students are trans- 
ported to appropriate sites, (Field 
Experience Office records, inter- 
views with Field Experience Office 
staffs faculty, school administrators 
and teachers, and students) 

Administrative procedures for the 
field experience component will be 
accurate, comprehensive, and 
prompt. 

Field sites will be appropriate for 
specific objectives. 

Effective communication will be 
maintained with school personnel. 

The program is commendable. 

Antecedent conditions in terms of 
administrative resources and pro- 
cedures and field sites are fully 
realized. 



dence and skill in the required 
activities. Students can themselves 
explain what they are doing and why 
(Transcripts; syllabi; course mate- 
rials; interviews with faculty and 
students; observations of field 
experience activities) 



Students will engage in a planned 
series of field experiences in appro- 
priate sites. 

Activities will promote students* 
mastery of theory and practice. 

The field experience component will 
be planned, supervised, and evalu- 
ated by faculty 

The program is commendable. 

Intended interactions are worth- 
while and occur with exceptional 
quality. 



The field experience component will 
inciC!'se markedly graduates' com- 
petence ua beftinning teachers. 



The progi'am is commendable, 

Positive, concrete outcomes occur 
and are evident to the program*s 
clients. 



65 



VI 

CONCLUSION 



Recently, the public has developed an intense interest in 
education and leveled harsh criticism at teacher prepa- 
ration. These jomplaints (some of them justified) must be 
addressed by teacher educators. However, in many cases, 
the public and its representatives speak from inadequate or misin- 
terpreted information. The situation provides an excellent opportu- 
nity for teacher educators to respond proactively by developing 
comprehensive systems for evaluating their pro^^rams. Teacher edu- 
cators must clarify for themselves, their clients, and the public exactly 
what their progi-ams do, under what constraints, and to what degi-ee 
of success. 

We encourage teacher educators to think of program evaluation 
as a normal, usefUl activity not restricted to evaluation specialists. 
We have attempted to provide not a narrow prescription, but rather 
a framework for thinking about evaluation in context, either to design 
a comprehensive, ongoing system or to improve an existing one. 

The public and other clients of teacher education have a right to 
current, accurate information about the quality of teacher prepara- 
tion programs. Teacher educators need the same information for self- 
improvement and also as concrete evidence of the contributions they 
have made to the preparation of effective school personnel. 
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